AA Interview Notes

1. AA Interview Notes James Hamilton JamesRH 2006-09-17

2. Agenda • Setting up the loop • Managing the loop • The Interview • Learning from the loop

3. Setting up the Loop • Make sure right folks on loop: • Will we have the info we need at the end of the day? • Experience of folks on the loop • One or, at the very most, two inexperienced interviewers • Make changes if needed • Don't just "accept" the loop • If two groups involved, are there enough from both teams? • 2 slots, one of which is lunch won't cut it • Read the resume, does it match the job? • Are there others it matches more closely

4. Managing the Loop: Feedback • If no feedback within an hour, complain to loop • Publicly whack those that send in crappy feedback: • "Nice guy" ... "really enjoyed talking to him“ • Make sure feedback has content -- don't accept 5 or 10 lines • If dev interview, insist on seeing code for all but lunch interview • Test interview requirements not much different • Read all feedback and learn who needs private advice • Must take a position: • “Tentative hire” is just wrong ... call it publicly • Insist on taking a position ... must start with "hire" or "no hire" • If uncertain, no hire • Calibrate interviews to individual being hired: • Insist on deep knowledge from senior folks • Expect core skills and potential from university hires • Correct quickly if questions are not at right level

5. Managing the Loop: Adapt • If failing as dev, for example, consider other roles • Some of our best PMs & Testers were once Devs • We paid to get candidate in so invest in finding potential • If candidate failing, jump in early • Call leaders of other roles & recruiting to redirect loop • It's your job to adapt the loop if it's not working • Under no circumstances send home w/o 3 interviews • If borderline, continue the loop • Leave candidate with a good experience • As AA try to see anyone who is borderline • Do the AA unless clearly not a hire • You’ll learn about both candidate & the interviewers

6. The Interview: Approach • Be casual and friendly and help the candidate relax. • The more relaxed they are, the more successful you'll be • You need details fast with only an hour • Best predictor of future performance is past results (outside the stock market ) • Weak at school, poor results in past jobs, not hitting the top of anything, yet suddenly a superstar is very rare • Core attributes shine through • Not everyone will excel in school but they better excel somewhere • If a "hard worker", find real examples • If an "excellent problem solver", get examples. • No examples almost assures it’s simply not true • Don’t waste time • Start and finish on time • Don’t waste much time on project or other introductory stuff • A hard interview followed by a job offer is motivational • Good candidates enjoy being challenged • You need the full hour to get deep insight into candidate

7. The Interview: Core Attributes • Recruiting encourages us to look for a broad set of core attributes • My focused set is a smaller subset: • Motivation, tools of the trade, & raw IQ • Recent style is each interviewer to focus on a core attribute or two • I’m not against this but don’t personally chose to do it • Raw IQ: from design questions & interactions during interview • Tools of the trade: I'll survey standard computer science education to make sure a candidate knows common algorithms, has appropriate education or equivalent work experience. • Be particularly careful of candidates that have a masters in computer science but an undergraduate in something like English lit. • Motivation: Discussions of past project, how they tackle problems, how they have worked through tough problems • Do they give up, did they hit the top, did they actually ship or switch projects early, etc., etc. • Track record of success: not all candidates succeed everywhere but, with motivation, they should excel in many areas

8. The Interview: Digging Deeper • Education: I typically do pass through education • checking for basics & quality • Experience: I typically do a pass through 2 or 3 of their major projects • I’m looking for two things: • Is it an accurate description of their past or someone they worked with? • I’m looking for interesting problems for later design questions • I'll chose an example from a past project and ask them for details • Assure myself first that they actually did the work • Then debate a few design decisions to see how they deal with technical conflict • Must be able to push back when on wrong track and step back when not • Change a design parameter and ask for a solution • What if it won't fit into memory, I need multi-user concurrent access, I want to reduce response time by a factor of 10, etc., etc. • Keep firing questions & design mods and see how they adapt • Coding: Ask a coding question where they don't already know the answer • You really can only afford 20 min on coding so it can't be too hard • As quickly as possible, syntactically correct, and correct algorithm • If dev/test ask coding questions in all interviews prior to AA • If any doubt, ask in AA as well

9. The Interview: Different Job Types • SDE/T: • I'll ask a simpler coding question or expect somewhat less but still want to see code. • I substitute design questions with testing questions. • Test a software component with which they are familiar, like a compiler symbol table, and instruct them to give me as many tests as possible as quickly as possible • If you don't get on the order of 15 or 20 pretty quickly, worry but keep trying. If no more, they don’t make the bar • PM: • I'll typically not ask a coding question, will still ask the same design question, and I'll go looking for customer focus in and around design questions. • Many questions on how to chose features, when to push a feature, challenge with schedule/feature trade-offs, challenge with late bugs and whether or not to fix. Basically, scenarios from various parts of the dev cycle to see how they react. • Ask questions from their past to understand how they will work with other groups in conflict. Could they evangelize a feature with other groups, work with other groups, reduce conflict, etc. • Present design sketches and ask them to criticize • Expect them to argue a point carefully, accurately, forcefully but not painfully. • Must be technical AND customer focused • How to influence a dev team without direct leadership position? • Architect: • Beware of someone who wants to be an architect for the titles sake • It’s not a role, it’s a broader, more senior version of one of our existing job types • Candidate must have been “in the trenches" & actually shipped successful production software • Can’t succeed without credibility and it won’t come if they haven’t “been there” • Ask for their principles of development and evidence of them being applied with success in past • Ask deep questions in their area of expertise and expect to get credible, articulate answers. • You should learn something from them • How do they deal with conflict? How do they influence the team?

10. Learning from the Loop • Introduce new interviewers by putting them on loops with good interviewers • They'll read the feedback and compare with their own • Don't introduce more than one new interviewer at a time • Go chat with new folks about what you learned and what they learned • helps them calibrate and learn to look deep • Those that aren't giving good feedback need to be reminded of their responsibilities • There are some folks that just never develop into good interviewers • don't put them on loops • Work with the HR team on who you actually want on loops • I like to get involved in who is on the loop, sometimes tuned to candidate and sometimes to the work load we're currently under • At bare min, make sure that the loop is appropriate and pass on changes to HR • Rarely, but occasionally, I'll send a "here's what we could do better" note to the loop or "it was a great loop and here's why" • Very rarely, when it's a borderline case, I won't make a hire/no-hire call that evening • We need quick answers so I only rarely delay when it appears the data is incomplete • Once I held a meeting the next day since the feedback ranged from "I would rather quit than work with this guy" to "we absolutely have to have him" • Keep an eye on how your hires do and learn from the result