On hiring - Choosing the right way to assess technical engineering skills in candidates

The tech industry is suffering from terrible technical interviews that lead to false negatives and hurt candidates. We can do better — here’s how.

I had completed a month-long job search in which I went through 75 interviews, resulting in 7 offers. The technical interviews I went through could be classified as good or terrible.

The good ones were great. They were engaging, fun, and left room to interact with people and go through the thought process and tradeoff discussions together. They felt like conversations and collaborative problem solving.

The terrible ones were truly terrible. Some asked me to invest five hours into take-home projects, despite the fact I had a job as well as an extensive portfolio of open-source side projects. Others ask me to spend hours to complete technical assessments that ended up not assessing anything related to the role. Others asked me to refactor or code, but then provided only skeleton boilerplate code that left little room to actually demonstrate engineering skills and technical knowledge.

As someone who has been on both sides of the hiring aisle many times, it pains me to see such terrible interviews being inflicted upon candidates and companies alike.

It does lasting damage

Bad interviews hurt candidates, who have to be exposed to stressful situations that leave them feeling defeated and dejected, even if they would make otherwise excellent hires.

They also hurt the companies themselves by extending the time it takes for them to hire, as well as leaving them with poor reputations amongst the candidates who have to endure it.

The process is broken

Companies simply don’t know what they are hiring for. Because of this, they use the wrong or ineffective technical assessments and evaluate the wrong skills.

Know the skills you’re hiring for

Did you know there’s differences between the following?

• Coding

• Programming

• Engineering

• Architecting

• Developing

Are you using the appropriate technical interview questions to identify the appropriate skills needed for the specific role you are hiring for?

Coding

Coding is the mechanical process of entering code instructions into the computer. Familiarity with language syntax, standard libraries, etc. are the skillsets involved here.

Examples:

If you’re asking someone to write a for-loop that prints the numbers 1 through 10, you’re testing their coding skills.

(1..10).each { |i| puts "#{i} " }# 1 2 3 4 5 6 7 8 9 10

Problems that assess coding skills often:

• Have a single method that has a deterministic result

• Have a well-defined or easily discoverable algorithm

• Have well-defined inputs

• Have well-defined outputs

The prompt might look like:

def fizzbuzz(n)
  # Print fizz if the answer is divisible by 10
  # Print buzz if the answer is divisible by 5
end

The above example has a very clear input, a very clear process, and a very clear output.

Even in cases where the algorithm is not explicitly defined, it is still relatively simple — there’s no complex calculations, forward-looking/backwards-looking logic, or gotchas.

What does this tell you about the candidate?

Candidates performance in coding gives a few clear signals:

• Are they aware of the language constructs available to them?

• Can they take a sequence of defined steps and input them into the computer?

• Can they translate the real-world steps into actual, working code?

However, there’s also some noise. Performance doesn’t necessarily indicate:

• Whether people can structure programs maintainably

• Whether people can implement business requirements

• Whether people can optimize systems

• Whether people can solve problems

• Whether people understand the technology, how to integrate with them, or the tools available to them

Coding assessments are great for filtering out people who don’t know anything about writing computer programs.

Programming

Programming is the act of creating a sequence of steps that can be translated into code to solve the problem at hand. Knowing algorithms, the various attributes of data structures, and problem analysis techniques are the name of the game here.

The solution space for a good programmer is optimal runtimes, minimal growth curves, and low memory and processor usage.

Also important to note is that there are two rather distinct types of programming problems.

Simpler programming problems like “check if this string is an anagram of another string” can be done with basic problem solving.

Advanced programming problems like “what is the minimum number of moves this knight must make before they have visited each square at least once?” requires a certain level of mathematical or algorithmic knowledge that many find difficult to derive or use in an interview setting.

Examples

Programming questions are often arbitrary problems with some sort of brute-force solution, and a hidden optimal solution.