Stop Testing Engineers on Memorization. Start Testing How They Think.

Here's a number that should bother every VP of Engineering with open headcount: 71% of engineering leaders globally now say AI is making it harder to assess candidates' technical skills. That's from a Karat survey of 400 engineering leaders across the U.S., India, and China. A few years ago, that figure was likely in the 20% to 30% range. It spiked to over 70% in the last two years.

The instinct is to blame cheating. Candidates using ChatGPT on take-homes. AI-generated code slipping through proctored screens. But according to Karat CEO Mohit Bhende, that's not the real problem. "It's because the fundamental job has changed, but technical interviews, as we know them, have not," he told IEEE-USA InSight.

That tracks with what we hear from IQTalent's technical recruiting team every week. The job changed. The interview didn't. And the gap between the two is where companies are losing their best candidates.

Two Clients, Two Approaches, One Clear Winner

Haley Crabiel, one of IQTalent's technical recruiters specializing in engineering hiring, described two clients she's worked with recently that illustrate the split perfectly.

"I was just on a client whose product was brilliant," she told us. "However, in their engineering technical interviews, they relied more on the memorization of code and being able to code in that direction."

Her current client takes the opposite approach. "The entire interview is: we want to see how you prompt AI," she said. "These engineers come in, and we know they're a good engineer because they're able to be strategic thinkers. They tell AI what to write. And then they still need to explain why they prompted it that way, what the outcome is, what they could have done differently, how that code is going to fix the problem."

The difference in what each approach is selecting for is stark. One tests recall under pressure. The other tests judgment, reasoning, and the ability to use AI as a tool, which is how 63% of professional developers already work, according to GitClear research. By 2028, Gartner projects that number will hit 75% of all enterprise software engineers.

Crabiel put it in terms anyone can understand. "Instead of understanding why two plus two is four, they're just saying two plus two is four. That's what they remember. That's it." The memorization-based interview selects for people who can perform under artificial constraints. The AI-fluency interview selects for people who can produce under real ones.

The GPS Analogy (And Why It Sticks)

Crabiel has a way of framing this that resonates with every hiring manager we've shared it with.

"It's like a driving test," she said. "Why would you say, go to the Walmart down the street, you have no idea how to get there, don't use GPS though? In their everyday life, they'd be able to use GPS. Why even test them on that?"

The question isn't whether the candidate can navigate without tools. The question is whether they can navigate well with them. "Are they choosing the longest way to get there? Are they choosing the most efficient way? There are things that still go into testing people on their abilities with AI without removing what they know."

This isn't a fringe position anymore. According to IEEE-USA reporting, slightly more than 25% of employers now allow AI use in technical interviews, and that figure is expected to reach 50%. Canva's engineering team publicly redesigned their entire technical interview process around what they call "AI-Assisted Coding," replacing traditional computer science fundamentals screens for backend and frontend roles. Their pilot found that the most successful candidates didn't just prompt AI and accept the output. They asked clarifying questions, used AI strategically for defined subtasks, critically reviewed generated code, and demonstrated strong debugging skills when AI solutions had issues.

That's not a lower bar. That's a different bar. And 73% of engineering leaders say a strong engineer is now worth at least 3x their total compensation, up from 60% the year before. When the stakes per hire are that high, you need the interview to predict performance. Memorization doesn't do that anymore.

What an AI-Fluency Interview Looks Like

Based on what Crabiel describes from her current client and what companies like Canva and HackerRank are implementing, the AI-fluency interview has four components:

Prompt. The candidate is given a real problem, not a toy problem, and asked to prompt AI to generate a solution. The complexity matters. As Canva's engineering blog notes, the problems need to be ambiguous and multi-layered enough that a single prompt won't solve them. Instead of "implement Conway's Game of Life," think "build a control system for managing aircraft takeoffs and landings at a busy airport."

Explain. The candidate explains why they prompted it that way. What assumptions did they make? What tradeoffs did they consider? Why did they structure the request the way they did? This is where engineering judgment surfaces, the thinking that AI can't do for you.

Evaluate. The candidate reviews the AI-generated output. What's wrong with it? What would they change? Can they debug it live? HackerRank's platform now captures AI-candidate interactions in real-time transcripts so interviewers can see how the candidate worked with the tool, not just the final product.

Iterate. The candidate describes what they'd do differently. What would the next prompt look like? How would they refine the approach? This tests the feedback loop that separates good engineers from great ones: the ability to learn from output, adjust, and improve.

That sequence tests understanding, not recall. It tests output quality, not performance under artificial constraints. And it mirrors how engineering work gets done in 2026, which means it predicts job performance better than any whiteboard exercise.

Q: How should companies redesign technical interviews for AI?

A: Forward-thinking engineering teams are replacing memorization-based coding screens with AI-fluency assessments. Candidates are given complex, ambiguous problems and asked to use AI tools to solve them, then explain their reasoning, evaluate the output, and iterate. Companies like Canva have already implemented this model, and IQTalent's CYBORG framework helps recruiting teams determine where AI belongs in the assessment process and where human evaluation is essential. The result is interviews that predict real-world engineering performance, not test-taking ability.

Why Companies Hesitate (And Why the Cost of Waiting Is Higher)

Crabiel is empathetic about why companies resist the change. "It's a whole restructure," she said. "We're redesigning this. That's also what's scary to companies: we've done this for so long and it's gotten us pretty far. We've got great technology teams and great engineers. Why would we change it?"

The answer is in the data. Job postings requiring AI skills nearly doubled from just over 5% in 2024 to more than 9% in 2025, according to CIO research cited by Atrium. AI-related job postings surged 163% year over year. The candidates who will thrive in these roles are the ones who work with AI fluently. An interview that bans AI is now selecting against that fluency.

And the redesign isn't as daunting as it looks. Bhende, the Karat CEO, recommends three steps: reflect on what skills the role requires (not what skills you've always tested), redesign the interview around real-world simulations relevant to AI-enabled work, and retrain interviewers to use a collaborative model where they read cues, provide hints, and jointly solve problems with candidates, rather than the traditional input-output questions format.

"Allowing AI is a new process, and it requires a holistic redesign of your interview framework and approach to doing assessments," Bhende told IEEE-USA. That redesign takes time. But the alternative, continuing to hire based on an obsolete signal, costs more in bad hires, slow ramp times, and engineering teams that can't keep pace.

Where to Start

If you're a Recruiting Operations Leader or VP of Engineering wondering how to make this shift, here's a practical starting point:

Pick one role and pilot. Don't overhaul every interview at once. Choose a high-volume engineering role, redesign the technical screen to allow AI, and measure whether offer-acceptance rate, time-to-productivity, and hiring manager satisfaction improve. Teams using structured, AI-supported interviews see 24 to 30% higher assessment consistency, according to Harvard Business Review research. You'll see the signal fast.

Define what "good" looks like with AI. Canva's engineering team found that the best candidates used AI for well-defined subtasks while maintaining control of the overall solution. That's your rubric. Not "did they use AI" but "did they use it well."

Tell candidates up front. Crabiel includes a note in what she calls her "interview kits" that AI is permitted. "But what does that mean?" she said. "I think even I'm a little hesitant on how to explain what that means." Clarity matters. Spell out what tools are allowed, what you're evaluating, and what the candidate should expect. 79% of candidates want transparency when AI is used in hiring, according to HireVue research. Give it to them.

Bring your recruiters into the redesign. Crabiel's point is worth repeating: "Recruiters can be used as those consultants, but recruiters could also be educated on how to consult the client and the candidate themselves." Your Recruiting Operations team sits between the hiring manager and the candidate. They see both sides. Include them in the conversation about what the interview should measure.

As Crabiel put it: "I think that's how it's going to change, people really start implementing this to test output and effectiveness." The companies that make that shift now will hire better engineers. The ones that wait will keep testing for skills their engineers stopped using a year ago. IQTalent's technical recruiting team supports engineering hiring across global markets, so the pattern holds whether your eng team is in Austin, London, or Bangalore.