I believe we have built a reasonable and consistent set of best practices for on site interviews at the Rackspace San Francisco office. We learned the initial structure from the Rackspace office in Blacksburg. One of the aspects I like about our process is that it can work for both technical and non-technical roles. We believe it isn't perfect and try to iterate on improvements. I would love to hear from other companies about their interview process and ideas for how to improve ours.

I am only describing the on site interview structure in this post, and not all aspects of the hiring process.

Note: We are always iterating on our interviews. We also do not have top down approach for the whole company. If you interview at other Rackspace locations or in the future, don't fret if the process is different.

Objectives

In a few hours interviews are attempting to ascertain a candidate's aptitude, fit, knowledge, potential and more. This is difficult. Human interactions are hard to measure, and the pressure of an interview does not lend itself to consistent results. In our interviews we try to achieve the following goals:

Consistency: We want to find potential Rackers on the first pass. A bad or inconsistent interview panel wastes both the time of a candidate and our teams.
Targeted: We want to make sure we are accomplishing specific goals in each part of the interview. Interviews are always too short, but if all interviewers ask the same question, we will not have a well rounded understanding of the canidate.
Culture, not just code: Rackspace is well known for it's distinctive culture. We want to expose candidates to this early and often.
Flexible for multiple roles: I believe interviewing a Product Manager under the same general structure as a Software Developer leads to teams being more effective in their execution of the interview.
Highlight the importance of interviews: Through structure we force our teams to dedicate time and focus to interviews. Interviews take significant time from busy people, but we do not want to reduce the quality of the interviews or produce inconsistent results.
Trained: We have an internal class called "H.I.R.E" which covers this hiring process, and educates potential interviewers.

The "Prehuddle"

The day before an on-site interview we schedule a 15-30 minute meeting with all of our interviewers. The hiring manager drives the meeting and supplies the interviewers with an interview guide that changes for each position. Here is a recent example of an interview guide used by Mailgun (PDF) The objective is to outline in writing the position we are hiring for, how they will fit into the team, and what each interview panel is trying to achieve. The Prehuddle gives time for interviewers to ask questions to the hiring manager so that they can come prepared to their interview panels. It also gives interviewers time to coordinate on who is covering each topic. We want to avoid situations where one interview panels assume another panel will ask certain questions.

On Site Interview Day

On the day of the interview we begin by giving the candidate a short tour of the office, ending in the conference room that will be used for the interview. On a white board in the conference room we have the schedule for the rest of the day written up. We try to keep the candidate in the same conference room for the whole day, to avoid losing 10 minutes of every panel to moving or finding the candidate.

An example schedule for a Software Development position:

10:00 to 11:00: Background and context
11:00 to 12:00: Computer Science and Algorithms
12:00 to 1:00: Lunch
1:00 to 2:00: Distributed Systems
2:00 to 3:00: Linux Systems

The interview consists of 4-5 panels of 2 interviewers. Each panel is generally 1 hour long. We consider it a best practice to also schedule an informal lunch with 2 more Rackers in-between the panels, but depending on the times of the panels this doesn't always happen. After the panel interviews, we want the hiring manager have few final words with the candidate and to escort the candidate out of the office.

Interview Panel Selection

When selecting interviewers for a panel, we consider the following:

Cross-Team: We want to have at least 2 interviewers who are on different teams than the hiring team. This encourages cross pollination of best practices and gives teams more context on the quality of their candidate.
Experience levels and time at company: We try to mix new hires in with long time Rackers to encourage learning by new hires.
Always someone in the room: We want to avoid on site interviews over video conference units. Because we are a remote office, some teams and roles will interact cross-location, or interviewers are traveling and a VC interview is unavoidable. In these situations we pair the remote interviewer with a local interview partner.

Interview Questions

We encourage our interviewers to use what we call the "SARA" model when interviewing candidates. SARA stands for Situations, Actions, Results, and Application. SARA is similar to a STAR or SOARA interview technique.
In the past we have tried to build a repository of questions or problems for interviewers to share with each other, but this has generally fallen out of our active practices. We have also tried code reviews, pair programming and other structures but have not been happy with the execution.
We have discouraged "Google Style" questions, but don't provide enough training or guidance on what the alternative is. This is an area I believe we must iterate and improve upon.

Feedback Session

We try to schedule a feedback session immediately after the candidate has left. Waiting until the next day will dull memories. We assemble all of the interviewers, and the hiring manager drives the meeting. We have the interviewers recall their interview in reverse seniority, with the hiring manager going last. Each interviewer has the floor for 2-3 minutes. Clarifying questions can be asked by other interviewers. After all of the other interviewers, the hiring manger shares their thoughts and the hiring manager then asks for any final conversation. Once this is done, we conduct an anonymous vote:

+2 Hard Yes
+1 Soft Yes
-2 Soft No
-4 Hard No

If the total is positive the hiring manager has the option to continue with the candidate. The hiring manager may still do other things like referral checking. If the total is zero or negative we will not hire the candidate.

Future Iteration Ideas

Here are some ideas I have been thinking about for continuing to iterate on our interviews:

Collaborative Coding: We currently use a white board for most of the coding exercises. I am not satisfied with this because it penalizes different ways of thinking and communicating. I would like to experiment with using a tool like Floobits. Floobits is more interesting than straight pair programming, as it could enable candidates to use their native editor.
Mixed length panels: I would like to experiment with making some panels 1 hour 30 minutes long, and others only 30 minutes. I believe this would allow stronger and deeper opinions about a candidate in these longer sessions, and the shorter panels would focus on smaller topics and meeting potential team members.

BYOT