We shouldn’t need to code in onsite interviews. However, in a sea of resumes, we need some kind of signal to understand how well a candidate understands the job they are applying for. Lacking any better way to answer “do you understand the concept of programming”, we’re left with technical interviews. We use these questions because we need to establish a baseline for programming competency.
As a company increases in size, our definition of competency changes. This is because as engineers, we are always told to hire smart people. In response, our once straightforward baseline questions evolve to look for smarter and smarter developers. The cumulation of this process is a complete migration from “show me something you’ve built” to “reverse a linked list”. As we try and find harder questions for candidates, we look online for guidance. We go to Google and we come back with knapsacks.
The Knapsack Problem illustrates everything that is wrong with technical interviews and how we conduct them today.
Given a knapsack size of “n”, what is the most valuable combination of items you can place in the knapsack without exceeding the weight limit?
Engineers love this shit.
Hell, the Knapsack Problem even has one true solution and “hints” written by smart people that you can copy/paste right into your company wiki.
Everything about the Knapsack Problem is flawed. When we pick this question, we do it ignorant of why it was a “hard” problem to begin with.
The Knapsack Problem above is known as a Dynamic Programming (or DP) problem. DP was conceived in 1953 by Richard Bellman as a way to break a large problem up into smaller sub problems. However, it wasn’t until the late 1960’s that Viterbi as well as Hart, Nilsson, and Raphael created algorithms that applied DP theory to real world problems. It took over 15 years to go from an idea to an algorithm, 15 years by some of the smartest people in our field.
And we expect someone who doesn’t recall the solution from memory to condense 15 years of computer science into a 45 minute window via clever hints.
That’s not just hard, that’s unrealistic and unfair. The only way you could complete the programming activity in time is strict recall. This places the Knapsack Problem and its ilk in the realm of what I’ve been calling Eureka Problems; named for the required “A-Ha” someone has to have in the interview to stand a chance.
The worst programming problems for technical interviews are in the bucket of Eureka Problems. If you don’t know what a Eureka Problem feels like, here are a few technical interview questions.
For these problems, it’s possible to develop a solution to the problem. Anyone who has done a technical interview recently smells something missing: constraints. With only a few more words, these questions become Eureka Problems, containing one true answer.
The restrictions force you to have a aptly named Eureka Moment for these problems. Like the Knapsack Problem above, you’re asked to condense the thinking behind a decade or more of computer science into whatever time remains in the interview.
These Eureka Problems are so ingrained in our interview culture that there are entire books written on coding interviews. That’s right. Entire books center around recognizing the 10 most common Eureka Problems and the 10–15 variants of each. Those questions above are all in the book, along with the Knapsack Problem from higher up.
Once you add the restrictions, you are no longer assessing the candidate’s ability to do the job asked of them. Instead, you’ve created a question that is assessing the candidate’s memorization and recall ability. That’s not what we want to know at all.
“Do I want to be on the receiving end of this person’s emails, code, & bug reports?”
This is the question you probably want to answer if you care about the software you are creating. It’s also easier to check by asking questions closer to the job at hand. If he is doing browser code, make a multi-column layout. If she is an application developer, have her call a mock of your API and do something with the data.
We say we want a holistic developer who sees the big picture. Someone who asks questions about the business, how it all fits together, and sees opportunity in all the different parts of the role. None of these traits come out in asking someone to recreate a pathfinding algorithm.
Let’s ask these instead, for example.
If you can’t get away from your attraction to classic Computer Science interview questions, consider accepting the less than perfect solution. You’re hiring someone to solve problems and write code, not to design algorithms.
“Do I want to be on the receiving end of this person’s emails, code, & bug reports?” Because that’s the question you need to be answering at the end of the interview.