```yaml
---
# This frontmatter will be used by CodeDrift
slug: on-take-home-coding
published: 2015-03-19T00:00:00.000Z
description: I'm Sorry, I Won't do Your Take Home Coding Exercise
changelog:
  - on: 2020-02-23
    note: This was rescued from the archives of the old [felocity.org](https://web.archive.org/web/20050507121849/http://www.felocity.org/journal). As a result, many links in this piece were left pointing to archive.org destinations.
  - on: 2020-03-15
    note: I'm more conflicted on this original piece in light of the pandemic. Interviewing against a dry erase board was already a bad measure; adding a remote code session just increases the number of things that can go wrong exponentially. That said, I think giving someone an activity they can perform in an _hour_ and having the candidate share their screen is a suitable alternative. If you absolutely must give people take home exercises, for whatever reason, consider compensating them for their time.
---
```

The instant we got rid of our "take home exercise" at LinkedIn, we saw an increase in the number of qualified candidates engaging in our interview process. We seem to be going against the grain here, as many large companies seem to want a "take home coding exercise" as part of the process before candidates even move into a more formal interview. I'm here to hopefully stop you, the reader, from falling into this trap.

## Noise, Not Signal

I totally understand the company side of this; take home activities offer a really effective way to get the candidate producing code which can be screened at the luxury of the company. You can save your already hired employee sometime by only interviewing people who have produced code that meets a minimum bar. You can even use this code during the on-site interview for follow up questions and diving deeper into the work in order to understand the candidate's thought process. By raising the barrier to entry, you've also ensured only people who really want to work at your company will proceed with the interview process.

What nobody counted on was the Internet. For all the time spent on building a question, one quick post to a site like Glassdoor can remove all of the value the company hoped to gain from a take home exercise. Building solid, reliable, and consistently measurable take home activities for candidates takes a very large and constant investment of time. And all of this before a single exercise has actually come back to your company.

If the interviewer thinks the code is flawless, then the question is probably on the Internet in some form. If the code was absolutely terrible, chances are that the recruiter wouldn't have provided the take home activity in the first place. This leaves everything in between, where an interview is still required in order to assess capability.

## The Candidate's Priorities

Things are even worse when one considers the other side of the interviewing relationship. Sending a take home activity to an engineer sends a very clear message that your company doesn't value their time. "Please build for me this fictitious thing", you say. "Complete this task, and we may let you talk to an engineer." That's a large hurdle to someone who doesn't even know if they would really like to be a part of your company in the first place.

Passive candidates don't have time, as they were quite happy with what they are doing currently(hence passive). These candidates often are spending their non-work time having families, pursuing hobbies, or contributing to open source projects. For these people, nothing is a bigger buzzkill than an activity that offers nothing to the candidate other than more of the same work they do.

Active candidates have the time. Since they are actively looking for a new opportunity, it's safe to say there's definitely more than one company they are talking to. In this scenario, asking the candidate to take 3 days for a coding activity is three days everyone else is actually talking to the candidate, assessing fit, and selling the candidate on the awesomeness of their respective companies. In these scenarios where an active candidate hedges their bets, take home activities rarely come back.

There are thousands of opportunities in our industry. When a company sends the "we don't trust you, so prove yourself" email, that company gets mentally shuffled lower in the queue for both active and passive candidates alike.

## The Work is Out There

It doesn't have to be like this. Our industry has already built LinkedIn, Stack Overflow, GitHub, BitBucket, TopCoder, and hundreds of other places where people can show their work. As a recruiter or engineer, ask them for code they would like to share. If there is no public samples available or offered, then it's worth sitting the candidate down with an engineer to perform a small directed coding task. Not only will the company get a chance to see the candidate's development process, the candidate will have the opportunity to learn more about what it is like to work with the team.

As for me, I will likely pass on your take home coding exercise. I hope my work can stand on its own.

I'm more conflicted on this original piece in light of the pandemic. Interviewing against a dry erase board was already a bad measure; adding a remote code session just increases the number of things that can go wrong exponentially. That said, I think giving someone an activity they can perform in an _hour_ and having the candidate share their screen is a suitable alternative. If you absolutely must give people take home exercises, for whatever reason, consider compensating them for their time.

Document

This was rescued from the archives of the old [felocity.org](https://web.archive.org/web/20050507121849/http://www.felocity.org/journal). As a result, many links in this piece were left pointing to archive.org destinations.

I'm Sorry, I Won't do Your Take Home Coding Exercise

On Take-home Coding

```yaml
---
slug: blind-references
published: 2017-12-09T00:00:00.000Z
description: or... Don't ask random strangers for their "back channel" opinion
changelog: ~
---
```

When faced with a difficult hiring decision, the least productive thing you can do is begin reaching out to your network in search of an "unvarnished" view of a candidate. These blind, or off-list, reference checks are both useless and dangerous.

*"He always left early to do stuff with his kids."*

*"Sally's bossy. It was always 'that time' for her."*

*"Jon was easier to work with once he started taking stuff for his anxiety."*

## The Danger of Off-List References

Let's get the dangerous part out of the way first. In fifteen years of working with companies, not a single blind reference check was ever suggested as a way to transform a "no hire" decision into a "hire" decision. That means that every blind reference check we do is actively looking for a reason not to hire someone. The repercussions of asking strangers for negative feedback are horrifying. Take a look at the above statements again.

Those are **actual blind reference checks**, sitting in three separate ATS systems, by three different companies. Protected categories extend to all your reference checks, but your premise for blind reference checks is going to naturally unearth vendettas, grudges, and other defamatory statements.

Or worse, your blind reference will go ruin your candidate's career. Consider Greg:

> I applied to WidgetSoft back in 2011 and got to the final stage of the interview. The hiring manager saw that I was connected on LinkedIn to Danny, a coworker from a few jobs ago. What the hiring manager didn't know was Danny and my boss were frat brothers. I'm sure Danny said something, because I was fired the next day. I "already had a foot out the door," according to my boss.

What was actually said came out, and it was settled before trial. For the curious, Greg didn't take the job.

So here's a better solution. If you're going to ask for an off-list reference, it's a "no hire" decision. It's that simple. You're done. We can all go back to interviewing candidates on their merits.

Every time you get the urge to make a blind reference check, it reveals a weakness in your interviews. Fix those, and the you'll get the answers you're looking for.

## Fix Your Interview

If at any point you or anyone in your company considers reaching out to their network for a blind reference check, stop the transaction immediately and write on the nearest writable surface. Your goal is to capture the raw thoughts and motivations behind wanting to check other references.

- "They seem to be dispassionate about the product"
- "I think they are a [brilliant jerk](http://www.brendangregg.com/blog/2017-11-13/brilliant-jerks.html)"
- "I wasn't convinced they are a strong coder"

These may be gaps in your interview process. Now is the time to ask the team, "what should we ask to dig into this issue deeper next time?" With a few tweaks to an existing interview or a small adjustment to a process, you can begin addressing these gaps.

Don't be surprised if you encounter a lot of resistance the first time you ask everyone to slam on the brakes. If your company habitually checks off-list references, folks have likely become comfortable with their flawed process. Since blind reference checks aren't done to overturn a unanimous "no hire" decision, that leaves two routes to explore:

**If the team was all "hire" decisions**, then why do we need to ask more folks outside of the interview process? What is it we couldn't glean from our own process? Why don't we trust our interviewers?

**If the team was split between "hire" and "no hire" decisions**, then why wouldn't we respect our interviewers and default to "no hire"? Are we just looking for confirmation bias?

## On-List References Still Have Value

All of this doesn't mean you ignore the references provided. While off-list references have significant dangers, on-list references can provide valuable insights into how to effectively onboard, manage, and grow your employee. Know what you are looking for in your references, and ask for references who can speak to the areas you want to explore. The following three individuals are a good place to begin.

- Someone who managed the candidate directly. This person will help you learn which relationships your new employee will need to prioritize building.
- Someone who was peer to the candidate. This will help you or their direct manager ensure the new employee is in an environment that is set up for success.
- (For Managers) Someone who the candidate managed. This will help you understand both the new employee's management style and the the support they'll need to be successful.

When it comes to questions, Sean Falconer has done tremendous work with his article ["Best Questions to Ask References"](http://blog.proven.com/questions-to-ask-a-reference). Not only is it a comprehensive list of questions, but includes the all-important "why" you'd want to ask these questions in the first place.

If you find yourself needing more references than were provided, the best thing you can do is be honest with the candidate. Your honesty will yield better references, and the candidate has the opportunity to provide individuals who can actually speak to the areas you want to explore.


or... Don't ask random strangers for their "back channel" opinion

strong

blockquote

Link

Blind References

```yaml
---
slug: coding-interviews
published: 2015-06-22T00:00:00.000Z
description: A rant on interviews that don't assess anything useful
changelog: ~
---
```

We shouldn't need to code in onsite interviews. However, in a sea of resumes, we need some kind of signal to understand how well a candidate understands the job they are applying for. Lacking any better way to answer "do you understand the concept of programming", we're left with technical interviews. We use these questions because we need to establish a baseline for programming competency.

As a company increases in size, our definition of competency changes. This is because as engineers, we are always told to [hire smart people](http://www.joelonsoftware.com/articles/GuerrillaInterviewing3.html). In response, our once straightforward baseline questions evolve to look for smarter and smarter developers. The cumulation of this process is a complete migration from "show me something you've built" to "reverse a linked list". As we try and find harder questions for candidates, we look online for guidance. We go to Google and we come back with knapsacks.

## Knapsacks in the Workplace

The Knapsack Problem illustrates everything that is wrong with technical interviews and how we conduct them today.

> Given a knapsack size of "n", what is the most valuable combination of items you can place in the knapsack without exceeding the weight limit?

![A knapsack showing a capacity of 15kg, surrounded by several generic blocks with various weights and prices](https://user-images.githubusercontent.com/1795/146829910-46154c35-282e-4724-85dd-490cfe2e0d52.png)
_Knapsack Problem Illustrated by Dake ([wikimedia commons](http://en.wikipedia.org/wiki/File:Knapsack.svg))_

Engineers love this shit.

- The basic implementation isn't "optimal"
- To do it "right" is "hard"
- The solutions on the internet are generally trusted

Hell, the Knapsack Problem even has one true solution and "hints" written by smart people that you can copy/paste right into your company wiki.

Everything about the Knapsack Problem is flawed. When we pick this question, we do it ignorant of why it was a "hard" problem to begin with.

The Knapsack Problem above is known as a Dynamic Programming (or DP) problem. DP was conceived in [1953 by Richard Bellman](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC528332/pdf/pnas00713-0067.pdf) as a way to break a large problem up into smaller sub problems. However, it wasn't until the late 1960's that [Viterbi](http://ieeexplore.ieee.org/xpl/login.jsp?tp=&amp;arnumber=1054010&amp;url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D1054010) as well as [Hart, Nilsson, and Raphael](http://ieeexplore.ieee.org/xpl/login.jsp?tp=&amp;arnumber=4082128&amp;url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D4082128) created algorithms that applied DP theory to real world problems. It took over 15 years to go from an idea to an algorithm, 15 years by some of the smartest people in our field.

**And we expect someone who doesn't recall the solution from memory to condense 15 years of computer science into a 45 minute window via clever hints.**

That's not just hard, that's unrealistic and unfair. The only way you could complete the programming activity in time is strict recall. This places the Knapsack Problem and its ilk in the realm of what I've been calling Eureka Problems; named for the required "A-Ha" someone has to have in the interview to stand a chance.

## Eureka Problems

The worst programming problems for technical interviews are in the bucket of Eureka Problems. If you don't know what a Eureka Problem feels like, here are a few technical interview questions.

1. Given an array of numbers, find the array range that contains the largest possible sum.
2. Given a grid and a robot at point [X1, Y1], write a solution that calculates the path to its destination [X2, Y2].
3. Given a linked list, write a solution that detects if the linked list contains a loop.

For these problems, it's possible to develop a solution to the problem. Anyone who has done a technical interview recently smells something missing: constraints. With **only a few more words,** these questions become Eureka Problems, containing one true answer.

1. Given an array of numbers, find the array range that contains the largest possible sum **in O(n) time**. (Solved using Kadane's algorithm)
2. Given a grid and a robot at point [X1, Y1], write a solution that calculates the **shortest** path to its destination [X2, Y2]. (Solved using A* search algorithm)
3. Given a linked list, write a solution that detects if the linked list contains a loop **without modifying our objects**? (Solved using Floyd's cycle-finding algorithm)

The restrictions force you to have a aptly named Eureka Moment for these problems. Like the Knapsack Problem above, you're asked to condense the thinking behind a decade or more of computer science into whatever time remains in the interview.

These Eureka Problems are so ingrained in our interview culture that there are [entire books written](http://www.amazon.com/Cracking-Coding-Interview-Programming-Questions/dp/098478280X) on coding interviews. That's right. Entire books center around recognizing the 10 most common Eureka Problems and the 10–15 variants of each. Those questions above are all in the book, along with the Knapsack Problem from higher up.

Once you add the restrictions, you are no longer assessing the candidate's ability to do the job asked of them. Instead, you've created a question that is assessing the candidate's memorization and recall ability. That's not what we want to know at all.

## Towards Pragmatism

"Do I want to be on the receiving end of this person's emails, code, & bug reports?"

This is the question you probably want to answer if you care about the software you are creating. It's also easier to check by asking questions closer to the job at hand. If he is doing browser code, make a multi-column layout. If she is an application developer, have her call a mock of your API and do something with the data.

We say we want a holistic developer who sees the big picture. Someone who asks questions about the business, how it all fits together, and sees opportunity in all the different parts of the role. None of these traits come out in asking someone to recreate a pathfinding algorithm.

Let's ask these instead, for example.

- Given this page on Stack Overflow (in an area your resume says you have experience in) can you help me figure out which answer is the right answer? How do we test it and find out?
- I have a design / product specification here. Can we walk through it and start on a basic proof of concept over the next 30 or so minutes?
- Let's take this code here that I wrote and isn't working. Can you sit over my shoulder as we debug this and help me create working code? We can also add some tests so I don't break it again.

If you can't get away from your attraction to classic Computer Science interview questions, consider accepting the less than perfect solution. You're hiring someone to solve problems and write code, not to design algorithms.
Remember.

"Do I want to be on the receiving end of this person's emails, code, & bug reports?" Because that's the question you need to be answering at the end of the interview.


A rant on interviews that don't assess anything useful

Coding Interviews

The process behind a successfully growing team

Hiring

2017

Blind References

2015

On Take-home Coding

Coding Interviews