How to assess a CTO candidate before you hire

The founders come out of the third interview with their CTO candidate sold. The candidate clearly knows their stuff, has run big teams and the conversation flowed from the first minute. Then we ask what they expect this person to have achieved by the end of year one, and the answer takes a while to come.

We see this a lot, and it is understandable. This person will shape the architecture, the culture of the engineering team, the pace of delivery and much of what gets said to investors. Even so, plenty of processes stop at checking that the candidate “knows a lot” and gets on with the founders. For a CTO, that is not enough to make the call.

First, work out which CTO you need

“CTO” means very different things depending on where the company is. At an early stage you want someone who still writes code, makes quick architecture calls and hires the first people on the team. In a scale-up you need someone who can take ten engineers to forty, bring in process without slowing everyone down and develop their first managers. In an established company the job is about budget, vendors, security, compliance and an organisation several layers deep.

Someone excellent at one of those stages can be the wrong hire at another. So before you speak to a single candidate, we suggest writing one page on what they should have achieved at six, twelve and twenty-four months. That page becomes your reference point for the whole process, and it also helps you describe the role honestly.

Technical judgement, which is not the same as trivia

At this level an algorithms test tells you very little. What you want to see is how they reason when they have to decide without all the information. In our searches we usually combine three things:

• Ask them to talk you through a major architecture decision they made: the options they had, what they gave up and what they would do differently now. Good answers include a mistake they own.
• A case built on your own platform. Share a real problem, under an NDA if needed (technical debt, a bottleneck, a migration that has been pending for months), and ask them to think out loud with you. Pay more attention to the questions they ask than to the answer they land on.
• A conversation with your engineers, if you have senior people. They will notice very quickly whether there is real depth or just vocabulary.

Leadership shows up in the examples

The people side usually counts for more than the technical side, and it is also the easiest to dress up in an interview. So keep bringing it back to specifics. Ask how they have hired, with what process, how many of those people are still there and why the others left. How they dealt with someone who was underperforming, or a conflict inside the team. Who they have helped grow into a role with real responsibility. And how they delivered bad news, a serious delay for instance, to the leadership team or a customer.

Listen out for the vague plural. “We used to”, “it was decided”, “the team went with”. When you hear it, ask what they personally did.

Do they speak the language of the business?

A CTO sits on the leadership team and needs to understand margins, commercial priorities and the link between infrastructure spend and growth. A simple way to test it is to give them situations. How would they decide between building something in-house and buying a tool? What would they do if sales promised a feature to a big customer without asking them first? How would they explain technical debt to a non-technical board?

If one of your investors has a technology background, a conversation with them can give you a second read on the candidate.

Chemistry with the founders, and then some

The relationship between CEO and CTO is one of the most intense in any company. Getting on well helps, but what really holds it together is having talked openly about the awkward things before anyone signs: how decisions get made when you disagree, how much say they will have over budget and hiring, and what happens if the company has to change direction.

A half-day working session on a real problem works very well here. You will see how they argue, how they listen and, above all, how they behave when they are wrong.

References and terms, well before the end

References chosen by the candidate will be positive almost by definition. To get something useful out of them, ask to speak to someone who reported to them as well as former bosses. Use open questions that force an example, such as “what kind of situation did they find hardest?” or “would you work with them again, and in what role?”. Then compare what you hear with what came up in the interviews. Always get the candidate’s consent before calling anyone, and never approach their current employer without permission.

Terms are similar. In a leadership role the package goes well beyond base salary: bonus, equity, retention or non-compete clauses, expectations about commitment. If all of that surfaces at the very end, it can sink a process that was going well. Cover the broad strokes in the first conversations and leave the contractual detail to your legal advisers.

Before you make the call

Check whether you can answer these with evidence rather than impressions:

1. Have they solved a problem like ours before, at a similar stage?
2. Have we seen them reason about our platform, not just theirs?
3. Do we have concrete examples of how they hire, develop and manage people?
4. Do we know how we would work together on the day we disagree?
5. Do the references back up what we saw in the interviews?

If the honest answer to any of them is “we think so”, you already know what the next step in the process is.

Good CTOs are rarely looking for a job, and reaching them takes discretion and a fair amount of patience. It is the kind of search we run at Mimmic Talent as part of our Executive Search work, so if you are facing this decision, tell us about it.