I constantly see the same complaints about job interviews:
- This is not what the job looks like!
- I’ve never used big-O notation in my career.
- Have you ever been asked to reverse a linked list at work?
But those can be good software engineering interviews. The industry has become much better than it was when the complaints were:
- Who cares how many ping pong balls fit in a 747?
- What do blenders and miniaturization have to do with software engineering?
- Why would you put the switches downstairs and the lights upstairs?
If none of these ring a bell, lucky you. Software engineering interviews used to be bad. They were bad in two different ways. Some were puzzles, the kind that give you an aha moment. They were supposed to test your thinking, but they didn’t. The other kind of bad interview was simply “having a chat.” These interviews are less consistent and more exposed to bias than structured ones. A recent meta-analytic revision ranked structured interviews highest among the procedures it reviewed. “Having a chat” helped create too many monocultural, non-diverse teams, often boys’ clubs.
The best modern interview processes are much more defensible than the old ones. That does not mean every company has one or that we should stop improving them. It does mean that the usual criticism has become too blunt: “this doesn’t look like the job” is not, by itself, a valid argument.
Two kinds of test
Let’s step away from software engineering for a moment. Interviews are tests, so it is useful to look at other forms of assessment and at the science of testing people for performance at work. One useful distinction is between a proxy test and a job simulation.
Simulations are what most people expect. They resemble the job without being the job. A good example is the firefighter Candidate Physical Ability Test (CPAT) used in the US and Canada: in a 50-pound vest, candidates climb stairs, drag a hose, carry equipment, raise a ladder, force a door, search, drag a dummy out, and pull down a ceiling with a pike pole. That is not a metaphor for firefighting. It is firefighting without the fire.
Something to keep in mind is that a simulation is not a random or average slice of the job. For a firefighter, a random 10 or 20 minutes might capture routine activity or downtime. That is not a good simulation. A good simulation samples important, demanding tasks identified from the job without reproducing every hazard. The CPAT, for example, combines eight tasks into a controlled pass/fail test with a 10-minute, 20-second cutoff. Hence firefighting… without the fire.
The second kind of test is a proxy test, which measures an underlying capability or skill. To join the U.S. Army’s 75th Ranger Regiment, a candidate must pass a fitness test that includes 41 push-ups in two minutes, a two-minute-and-35-second plank, a five-mile run in 40 minutes, and six pull-ups. Nobody thinks a combat mission is a timed sequence of gym exercises followed by a road race. The test is trying to establish whether a candidate has the strength and cardiovascular endurance for the parts of the job that suddenly become intensely physical.
Proxy tests exist because constructing a direct simulation is often too hard, cumbersome, or impossible. But that does not make them a simple shortcut. A plausible story about what a proxy measures is not enough: employers need evidence that the test, its scoring, and its cutoff measure something important to performance in the actual job. Selection guidance distinguishes criterion, content, and construct evidence for making that case.
Both ways of testing can be legitimate. Depending on what is being tested, an employer might use one, the other, or both. Neither is legitimate merely because it has the right label.
Now, in software engineering
Now let’s come back to software engineering interviews. Several interview formats are simulations:
- Pairing to solve a bug.
- Reviewing a pull request.
- Building something with AI-assisted coding.
In my experience, these draw the fewest complaints, but they cannot be an average slice of the job. They compress a job into the 30- to 45-minute window normally available for an interview. They aim to sample important work and score it consistently. Work-sample guidance also cautions that direct samples are most appropriate when candidates are expected to arrive with the necessary skills, rather than learn them on the job. We should also never grab whatever problem we are currently dealing with at work. Nobody wants to work for free, and scoring consistently over time would be impossible.
Then there are interviews that test by proxy. These are the ones that many candidates hate:
- System design: design Twitter.
- Data structures and algorithms: reverse a linked list.
- Machine learning: derive backpropagation for a neural network.
Designing Twitter is a proxy for recognizing a systems-design problem and tackling it properly. At work, an engineer is unlikely to sketch twenty parts of a global service in 30 minutes. They are more likely to make one deep decision. The interview tests whether the candidate can recognize that challenge and rise to it. When they cannot, we get unintended technical debt and broken systems.
Reversing a linked list is a proxy for algorithmic reasoning. Nobody should write that algorithm at work. But every system I have inherited contained code that executed in exponential time. An engineer had tested a piece of code with five samples and put it into production. On day one, everything was fine because there was little data. The engineer declared the task complete, moved on, and switched companies. By the time I arrived, the system was crumbling despite a scale that did not justify it.
Deriving backpropagation is a proxy for understanding the machinery underneath autograd. At work, frameworks usually calculate the gradients. But when training behaves unexpectedly, an engineer who understands how those gradients are produced can inspect the computation, identify where it went wrong, and troubleshoot it better than someone relying on blind trial and error around a black box.
There is a central missing piece in my argument: do these interviews, and the way we score them, translate to job performance? Unfortunately, I could not find public evidence that answers that question well. What I do know is that companies at Google’s scale are the ones that would have the data, and they abandoned brainteasers while retaining interviews on computer-science fundamentals. That is indirect evidence, though. I’d love to study this more deeply.
When everyone trains for the test
This is a separate dimension of the problem, but it affects everyone. By now, most engineers seeking a good job train specifically for interviews. That changes how scores should be interpreted.
The question is whether preparation improves the capability the job needs or only the candidate’s ability to pass this particular test. Someone may acquire a durable, job-relevant skill and genuinely cross the threshold. That is a success, not an artificial pass. Or they may acquire short-lived, test-specific familiarity and regress immediately afterward. The latter means the test is admitting people who have learned its tricks rather than demonstrated the capability it was meant to measure.
Sometimes, when I see people mechanically practice LeetCode instead of learning data structures and algorithms, I worry that they are learning the tricks rather than the skills. But I have no evidence for that. My experience of preparing for interviews was that some of my time went to refreshing (or learning) the fundamentals, and some went to learning a few tricks.
This is also an access problem. Practice and retesting do produce score gains, although that alone does not tell us whether prediction gets better or worse. If those gains come mainly from expensive coaching, repeat attempts, or knowledge of the exact format, the score will partly measure access to preparation rather than the capability the employer cares about. In a large operational study, employer-provided practice tests raised later scores, which is precisely why equal access to preparation matters.
In tech, access is not about access to information (which is publicly available on the internet). It means time and energy. An employer cannot control that. Those with more time have a privilege, and they tend to be those whose families can support them while they look for work. The only thing I can say to candidates is this: if you care about your career, invest in preparation as much as your resources allow. It is not a level playing field, but you can still improve your chances.
It gets worse!
There is another way in which interviews can be good but be perceived as bad: the number of rounds.
This is something candidates almost never hear. Often, one interview does not prove that a candidate failed, but it also fails to establish that they succeeded. When reviewing the outcome, notes, feedback, and recording, a decision-maker may find that the interview neither demonstrated that the candidate lacks a skill nor established that they possess it.
When that happens, my first approach is to look for evidence of that skill in the other interviews. This is where a battery of interviews can pay off. Multiple interviews are most useful when they gather job-related, reasonably independent evidence. Simply repeating a noisy exercise or having five people conduct the same unstructured chat does not ensure a better result. Sometimes I find the missing evidence, and sometimes I do not. Often, I wish we could book another interview. But an additional interview has such a bad reputation that it is a risky choice. Rejecting the candidate and moving on is, sadly, less risky.
Imagine someone posting this on LinkedIn: “They interviewed me five times and then they had the gumption to reject me.” I would not expect much outrage. Compare that with: “They interviewed me five times and then they had the gumption to ask for one more.” I can already see the comments: “They are not respecting your time,” “What else could they possibly learn?” and “They should start paying you for your time.”
The chance to interview again can be a net positive for the candidate when the missing evidence is specific and the extra round is bounded. An open-ended process that keeps discovering new requirements is not. Even a useful extra chance can feel, emotionally, like a net negative. I have been there: being told “one more” many times.
The test that annoys me: by experience
There is another kind of interview test: experience. It simply checks whether the candidate has solved the problem before, or at least a similar one. It is most clearly encapsulated in questions that start with “Tell me about a time when you…” These are extremely common in leadership interviews.
They annoy me because these tests mix several things in one question: what a candidate did, whether they had the opportunity to do it, how well they remember it, and how well they can explain it. Past behavior may be evidence that someone can do something again, but it does not establish that on its own. For example, in management, it is common to ask, “Have you managed 50 people or more before?” because the company has a team of 40 and hopes it will grow. It does not ask or test, “Can you manage 50 people?” Granted, that is a hard dimension to test for.
This is why, for leaders and managers, the evidence available for a next role is partly bounded by the opportunities their current and previous companies gave them. Choosing a job is therefore one of the most important career decisions they can make: it affects that job and every job afterwards. They can prepare to explain and analyze their experience better, but they cannot manufacture an opportunity they were never given. Contrast that with data structures and algorithms, and LeetCode starts to look more appealing.
I have seen these tests used widely in specialties outside technology, such as marketing and sales. But my experience there is limited.
Conclusion
Interviewing is not perfect, but the state of the art for engineers is pretty good. We should continue to improve it. Larger companies are the ones that can make the biggest difference. I hope they treat this as publishable material rather than a proprietary advantage. If they do not, the rest of us can only copy their practices and hope they are doing the science properly in secret.
One thing I have not yet addressed is AI. AI is disrupting all of these interviews, but it does not simply destroy proxies or require every simulation to become harder. It changes the competencies we need to measure. If engineers will use AI on the job, employers should test how they use it: how they prompt, inspect results, course-correct, and use skills and MCPs. But AI makes it extremely easy to cheat in proxy tests. That does not mean the candidate has passed the proxy, or that the proxy is unnecessary. If the soldier jumps on a bike when they are supposed to run five miles, that does not mean the job does not require high cardiovascular competence. I suspect we will see on-site interviews again to quash rampant cheating.
I’d love to be solving these kinds of problems at scale. This is the kind of thing that excites me about building teams.
