THE LEAFLET

July 03 2025

expressive vs impressive interviews, using biased LLMs to check bias, clarify decision rights

EXPRESSIVE VS IMPRESSIVE INTERVIEWS

Interviews are theaters of cognitive bias. Smart candidates have learned how to game them; smart interviewers are both aware of their own cognitive biases and believe, nonetheless, that their intuitions are trustworthy. Both parties set themselves up for disappointment.

So Ben and I recommend thinking of interviews as much as a chance for the candidate to learn about the firm as a chance for the firm to learn about the candidate. If you accept this premise, you design your hiring process as a clear and thorough expression of the culture, modes of problem solving, and rhythms of your organization rather than a narrow opportunity for the candidate to make an impression about who they will be on the job.

This expressive mandate reshapes the questions and tasks you give candidates. Instead of throwing out vague, open-ended prompts to see if they “get it”, you tell them directly what the “it” is that they are supposed to “get” - then ask them to do something with it. The conventional approach amounts to “gotcha” exercises that reward inside knowledge and pre-existing connections. If you replace “gotchas” with clear expressions of your values and challenging invitations to practice them, you get data that’s likely more valid and more relevant to their future performance. And you make jobs on your team accessible to people who may have higher upside – because of their mindsets and values alignment – than the insiders who already know which catchphrases make you nod and smile.

Ben instituted a strong version of this at the schools he led. Prospective teachers would plan and deliver a sample lesson for current students at the school. Ben transformed the sample lesson from a “gotcha” to an “around here, [you’ve] got to” with what followed. 

“At this school, we’re obsessed with growth. We care about it even more than absolute levels of achievement, and we care about those a lot. Everyone who works here gets a ton of feedback, and we expect them to seek it out, receive it with enthusiasm, and make best use of it. The feedback is a key driver of the growth we care about so much.”

“So we’re going to observe your sample lesson. Then we’re going to give you direct feedback on that lesson, just like we’ll do when you’re a real teacher here. Then - you’re going to do the sample lesson again. We’re looking to see how well you take the feedback. That matters to us even more than the overall quality of the two lessons. You should be looking at how this pattern feels to you.”

-eric

Read the rest here.

USING BIASED LLMs TO CHECK BIAS

Much has been made about potential biases in LLMs - ones that emerge by default from their training content and ones that might be deliberately coded into model weights. I take that critique seriously, and as a result consider the LLM’s take as a perspective - like that of a person, one that is tilted, imperfect, and useful, if for no other reason than it is not my own. 

In a recent hiring process, I interviewed a candidate 1:1 who made some strange choices in the conversation. I couldn’t tell if it was sheer nervousness or bad judgment on their part, but their performance landed badly with me. At the very least, the vibes were off. 

If I had simply trusted my gut, gone with my intuition, this would have been the end of the road for this candidate. We had a strong applicant pool and I was ready to screen them out at this early stage of the process. 

But I didn’t feel great about this call. So I typed up a summary of what I saw in the interview and how I interpreted it, then shared that with Claude and asked “what do you make of this?” 

Claude said: note it but don’t make too much of it. If you held aside this weirdness in the conversation, would you otherwise advance this person? For me, that was a yes. They had a record of achievement, had been a team leader in several phases of their career, and offered evidence of a growth mindset in practice. Good enough to move forward.

Interestingly, this candidate ended up making it to the final stage of a competitive, rigorous process. Through subsequent rounds, my view of this person changed so much that I was ready to offer them the job if our top candidate didn’t accept. In other words, they proved themselves to be fully qualified for the job, despite their uneven interview performance.

I was delighted and shaken by this. It was a good reminder of how fickle a 1:1 interview can be. And of how useful it is to blunt the bias of a person or tool by adding more perspectives rather than striving to purify any one of them.

-eric

Read the rest here.

CLARIFY DECISION RIGHTS FOR BETTER FOCUS

Delivering clarity in response to ambiguity is a real gift a leader can offer to their teams. One way I’ve seen great leaders do this is bringing subtext into text. Another good version is assigning decision rights and, in some cases, offering a fresh rationale for that assignment. In other words, who gets to make this call and why?

You may have good working policy on decision rights already. For good reason, you have already chosen a culture of permission or a culture of constraint. Even so, stuff comes up! Situations are weird. You can never pre-write the rulebook to account for every scenario. The person who normally would decide is on parental leave … while their team is being absorbed into another team … and some key board members expressed misgivings about the whole strategy both teams are being arranged to execute (or whatever).

I’m not arguing here for leaders to rule by fiat or for democracy by default. I’m saying - if you’re the leader, make it clear to the folks with a stake in the outcome who is deciding and how others’ input (of time, effort, viewpoint) will be weighed. This can dramatically lower the transaction cost of the decision. People spend less time and energy on petty politics (like driving, a thing most people think they’re more responsible and good at than they are) and more on achieving a good outcome. You resolve the question of process so everyone can put attention on the question of substance.

-eric

Read the rest here.

COMPELLING QUOTES

Family therapist Terry Real on firmness:

There is nothing that harshness does that loving firmness doesn’t do better.

Author Ted Chiang on determinism:

What if the experience of knowing the future changed a person? What if it evoked a sense of urgency, a sense of obligation to act precisely as she knew she would?

Novelist Hilary Mantel on promises:

That's the point of a promise, he thinks. It wouldn't have any value, if you could see what it would cost you when you made it.

Keep going, keep growing,

Ben & Eric