Saturday, November 16th, 2013
It's no secret that technical interviews are tough to get right. They're draining on the candidate and are a notoriously
difficult way to assess a person's technical qualifications. Over the last 8 years I've been heavily involved
in the interviewing processes at a number of different companies. I've given over 250 interviews, served on several hiring
committees, and screened countless resumes.
After a while I started to realize that many great candidates were incorrectly discarded by the process, while many less than
stellar candidates were managing to get through. If technical interviews are a key discriminator for the hiring process, they
do not appear to be very effective.
Let's Try Something New
The following process is far from perfect, but I believe it has merit. I've written this post to solicit feedback from readers,
so let me know what you think — particularly if you feel this is way off base or have encountered some of this in other
interview processes.
I believe this process is more accurate, effective, and comprehensive than several current methods, and I have used this process a few
times and received positive feedback (both from hiring committees and candidates).
Let's assume that you've already identified a promising candidate and have invited them on-site to interview.
Session One — Behavioral Assessment
The interview day begins with a one hour behavioral session with a relevant team member. The goal of this
session is to gauge the candidate's past behavior and use it as an indicator of potential future behavior.
The candidate should be asked probing questions about past challenges, situations, and decisions. The interviewer
should request specifics about the candidate's role in the situation, what the candidate did, how they affected the outcome,
and what they learned. Pay close attention to any inconsistencies and ask for clarification.
During this type of session it is common for interviewers to jot down notes on a laptop or tablet. This can have a distancing
effect, so opt for pen and paper instead. After the session is complete, the interviewer will produce a summary
mail that covers the contents of the session along with distilled thoughts about the candidate's behavioral fit.
Session Two — Technical Assessment
The second session focuses on assessing the coding abilities of the candidate. Candidates are requested to supply
source code, from one large project, a week before the interview date.
Several restrictions apply:
The candidate must be the primary and original author of the code.
The candidate must have the rights to distribute it.
The code must be of significant length.
If a candidate lacks a suitable project for this session, then a project may be provided for them to study.
While passionate engineers tend to work on side projects, not all engineers have the time to do so. It should not be
viewed as a red flag if a candidate lacks a suitable project for this session.
During this session the candidate is asked probing questions
about select portions of the code. Questions may involve design, functionality, optimization opportunities, data structures,
coding style, architecture, and algorithmic analysis. The candidate should be able to speak competently about every line of
code, and withstand even detailed questioning by the interviewer. A whiteboard is likely necessary during this session.
Depending upon the amount of time left in the hour, the candidate may also be asked to write alternative solutions to
problems presented in the example project.
Once session is complete, the interviewer will produce a summary mail that covers the contents of the session
along with distilled thoughts about the candidate's technical abilities.
Session Three — Social & Cultural Fit
The final one hour interview in this group is a lunch interview. This may be 1:1 or many:1. The goal of this section is to assess the personality fit. In similar fashion to previous sessions, the interviewer is responsible for producing a summary mail that covers the contents of the session along with thoughts about the candidate's social and cultural fit.
Session Four — Day Project
After lunch the candidate is directed to a conference room and given a 3 hour project to work on. This project must be
written, compiled, and tested within the time limit. The candidate is given a clean laptop with all the necessary tools
on it. The candidate is not provided Internet access, and the problem should be structured such that an online reference is
unnecessary.
The candidate is advised to spend at least 30 minutes designing the solution, and to leave at least 30 minutes after
completion for testing and debugging. A proctor is available during the first two hours of this session to answer
questions.
This session tests the candidate's execution abilities. The assigned project should be difficult enough to flex the
candidate and stress multiple areas of understanding. Top candidates should be able to complete the project (plus
optimizations), but should require the full three hours to do so.
Session Five — Communication Assessment
Once the three hours are over, the candidate is asked to present their solution and results to a small group of engineers.
The presentation will last 20 minutes, followed by 20 minutes of questions from the audience. If the candidate did not
arrive at a complete solution, they will simply present their progress.
The candidate should cover the initial problem, their solution, any problems encountered, and any other interesting
items (such as novel optimizations). If the candidate completed a working solution, then the audience should be
invited up to manually test the software.
This session further tests the candidate's communication skills. A crucial part of working in teams is having the ability
to understand colleagues as well as convey technical information accurately.
After the session is complete, the audience members will each produce a summary mail that covers the contents of the session
along with distilled thoughts about the candidate's execution, technical, communication and social abilities.
Session Six — Wrap Up
The final session is a quick wrap up between the hiring manager and the candidate. This session is dedicated to covering any questions the candidate might have about the day or the company. It is very important that candidates leave with a positive impression of the company, regardless of how they feel they performed. This session enables the interviewer to set expectations for a followup and close out the interivew day.
Hiring Committee
After each phase of the on-site interview, interviewers are expected to write up their results and deliver them to a hiring committee. This committee consists of employees who were not part of the on-site interview, and do not know the candidate personally. The hiring committee also receives all relevant information about the candidate, including letters of recommendation, the resume, and recommendation letters from existing employees at the company who know the candidate. It is then the role of the hiring committee to reach a hiring decision about the candidate.
Conclusion
The goal of a hiring process is to properly filter out inferior candidates, while selecting and hiring the most qualified ones. Unfortunately,
this is a very difficult process due to a variety of factors. Some candidates simply have bad days, while others get lucky and
manage to hide troubling technical or personality problems.
While the process outlined in this article undoubtedly has its flaws, I have found promise in its ability to uncover qualities or issues in
a candidate that would have otherwise gone completely unnoticed with other interview processes.
Thanks for reading! Please remember that I am interested in feedback on this, so if you feel that I'm completely wrong, or spot on, please
feel free to drop me a note!
