Technical interview as a service for 11 engineers in 8 weeks
A Series B SaaS company in San Francisco used The Cognitive as a technical interview as a service: live AI video interviews ran its first technical screens. Customer-reported: 11 engineers hired in 8 weeks, senior-engineer screening from 15 hours a week to 0, time to offer down from 49 to 16 days, and final-round pass rate from 28% to 71%.
What changed when the AI took over technical screens?
The first 3 rows are the customer's own figures. The last 2 are our arithmetic on them; the working is further down the page.
| Metric | Before | After | Change |
|---|---|---|---|
| Senior-engineer screening hours per week (customer-reported) | 15 | 0 | All 15 hours back |
| Time to offer (customer-reported) | 49 days | 16 days | 33 days shorter, 67% less |
| Final-round pass rate (customer-reported) | 28% | 71% | About 2.5 times as high |
| Final rounds that end in a no, per 100 (our arithmetic) | 72 | 29 | 43 fewer wasted final rounds |
| Senior hours returned over the 8 weeks (our arithmetic) | 0 | 120 | 15 x 8 weeks |
Figures reported by the customer in our case study record dated 4 August 2026.
What does technical interview as a service mean here?
Technical interview as a service Handing the first technical interview to an outside service, so in-house engineers stop running screens. Some services use freelance human interviewers. Here the service was software: a live, two-way AI video interview working from a rubric the team wrote.
Every applicant got a spoken technical interview with the AI. Engineers read reports and recordings instead of sitting in screens, and joined only for final rounds with candidates who had already cleared the bar.
Who is the customer, and what problem were they facing?
The customer is a Series B SaaS company in San Francisco, named in our record only as Series B SaaS Company. It had 4 open backend roles and more than 300 applicants for them. Each technical screen cost a senior engineer an hour, plus prep beforehand and a write-up afterwards.
The arithmetic explains the bind. Screening all 300+ applicants at an hour each would take 300+ senior-engineer hours before any prep or write-up. In practice the team was spending 15 senior hours a week on screens, and 72 of every 100 candidates who reached the final round still did not pass.
The team saw 2 options, both bad. It could burn sprint capacity on interviews, or filter hard on resumes and risk missing strong engineers who had not worked at well-known companies.
We stopped outsourcing our roadmap to our interview calendar. The AI does the screens; our engineers do the final rounds and the shipping. VP Engineering, Series B SaaS Company, San Francisco
What did the team try before AI technical interviews?
First they tightened recruiter screens and leaned harder on resume signals: well-known employers, familiar stacks, tidy career paths. The record's verdict is that the queue looked cleaner without getting better. Engineers still spent focus time checking basics, and the 28% pass rate showed the filter was not doing its job.
The stricter resume filter carried a second risk. A backend engineer from a small company or an unusual path could have exactly the skills the roles needed and still never reach a screen, because the resume did not look like the obvious safe pick. The team wanted a way to hear from those people without paying an engineer-hour for each one.
The wider market shows why the problem keeps growing. Ashby's 2026 Talent Trends data has technical hires taking 17.6 interviews and 23.3 hours of interview time each, up 52% in interviews since 2021, and Gem's 2026 benchmarks find only 8% of applicants pass initial screening (Ashby and Gem, read 5 October 2026). At this company, the screening share of those hours fell on senior engineers.
What did the backend rubric measure?
An example rubric: criteria and weights stay fixed for the role, and each one describes what a 1, a 3 and a 5 look like.
The rollout started with the team agreeing which evidence mattered for backend roles. In the record that meant system design and problem-solving, talked through live. The criteria, their weights and the 1, 3 and 5 descriptions went into the role, where both the interviewer and the scoring read them.
How did they run technical interviews as a service, step by step?
The setup dates from August 2026 and the product has changed since. These steps describe how the same approach runs in The Cognitive today.
- Agree the evidence with the engineers Before any invite, the hiring managers wrote down what a pass looks like for each backend role and turned it into weighted criteria. Our suggested backend weights: system design 30, debugging 25, data modelling 20, reliability thinking 15, communication 10.
- Set the role to 20 minutes A recruiter picks 10 or 20 minutes per role. 20 gives the AI room to follow a design answer 2 or 3 levels down. 20 minutes fits one design question with 2 follow-ups, one debugging scenario and up to 5 resume claims checked along the way.
- Invite every applicant, not a resume shortlist Import applicants from the ATS through 60+ ATS integrations or a CSV and bulk invite the whole batch. Candidates book their own slot without an account. Inviting everyone is what lets an engineer from a small company show the skill a resume filter would have hidden.
- The AI runs the technical screen live It asks design and problem-solving questions over video and follows up on each answer, probing tradeoffs, failure modes and what the candidate would measure. More backend prompts are on our [backend developer interview questions](/interview-questions/backend-developer) page.
- Engineers review reports, not calendars Each report gives 1 to 5 per criterion, a weighted score out of 100, overall written feedback, the transcript and the recording. Early reports were too long, so the team trimmed what reviewers saw. A review rule we suggest: one engineer reads the day's reports in a 30 minute block and watches recordings only where design and debugging scores disagree.
- Humans decide and run final rounds A hiring manager accepts or overrules the suggested verdict. Nothing is rejected automatically, and the ATS gets a note while stages stay where the team puts them. Use the final round for what a spoken interview cannot show, such as a pairing session or a code review exercise.
- Source for the hardest seat When one role draws thin applicants, search ~900M public profiles from a plain-English brief, reveal verified emails and phone numbers, and invite shortlisted engineers to the same AI interview. A brief like: backend engineer with Postgres and event queues at a B2B SaaS company, San Francisco or remote US.
What example questions suit an AI backend technical interview?
Examples you could add to a backend role. They are illustrations, not this customer's questions. The AI asks added questions in its own order and builds follow-ups from each answer.
Design a service that sends each customer a webhook when their invoice is paid. Talk me through the main parts.
Separates receiving the event from delivering it, uses a queue, plans retries with backoff, and says how a customer can tell whether delivery failed.
A customer's endpoint is down for 6 hours. What happens to their events?
P99 latency on one API endpoint doubled after yesterday's deploy. How do you find out why?
Compares what changed, checks traces and database query times before guessing, and decides early whether a rollback is the safer first move.
The deploy only changed a logging library. Where do you look now?
A table you own has grown to 500 million rows and one query is slowing down. What are your options?
Reads the query plan first, considers an index or a narrower query before sharding, and names the cost of each option for writes and operations.
Adding the index locks the table. How do you ship it safely?
A downstream payments API starts timing out for 20% of requests. What does your service do?
Sets timeouts and retries with limits, makes requests idempotent so retries cannot double charge, and degrades gracefully instead of failing every request.
How would you know the retries are making things worse?
Design the API for listing a customer's invoices when some customers have 2 million of them.
Chooses cursor pagination over offsets, explains why, and thinks about filtering, sorting and what happens when data changes between pages.
A client wants every invoice in one export. What do you offer instead?
Why did the final-round pass rate rise from 28% to 71%?
The record calls pass rate the honest measure of whether a screening step works, and the numbers back it up. At 28%, 72 of every 100 final rounds ended in a no, so most of the hours engineers spent in final rounds went on people the earlier steps should have caught. At 71%, only 29 of every 100 ended that way.
The reason is what reached the panel. Before, candidates got to the final round on resume signals and short recruiter screens. After, every finalist had already talked through design and debugging questions with the AI, and a hiring manager had read that report. Engineers met people who had shown the skill rather than people whose resume suggested it.
Time to offer fell from 49 days to 16 in the same period, 33 days or 67% faster, because the first round no longer waited for a free engineer slot. Ashby's 2026 data puts technical roles at about 48 days from application to accepted offer, so this team started right at the market norm and finished about a third of it.
The working, line by line
Inputs are from the customer's record, plus BLS pay and Ashby's benchmark. Everything after an equals sign is our arithmetic.
SENIOR-ENGINEER SCREENING HOURS Before 15 a week. After 0. 15 hours x 8 weeks = 120 hours returned. 120 x $64.44 (BLS median hourly pay) = about $7,733 of engineering salary. IF ENGINEERS HAD SCREENED EVERY APPLICANT 300+ applicants x 1 hour = 300+ hours, before prep and write-up. 300 x $64.44 = about $19,332. TIME TO OFFER Before 49 days. After 16 days. 49 minus 16 = 33 days shorter. 33 / 49 = 67% less time. FINAL-ROUND PASS RATE Before 28%. After 71%. 71 / 28 = about 2.5 times as high. Final rounds ending in a no, per 100: Before 100 minus 28 = 72 After 100 minus 71 = 29 72 minus 29 = 43 fewer per 100. PACE 11 engineers / 8 weeks = about 1.4 a week AGAINST ASHBY'S 2026 BENCHMARK 11 hires x 24.7 interview hours per engineering hire = about 272 hours 16 days to offer against about 48 = a third of the norm
What did engineers read instead of running screens?
An example interview report for an engineering role: a score per criterion, a weighted score out of 100 and resume claim checks.
The hiring manager's debate moved from whether a resume deserved a screen to what the candidate actually said. A reviewer reads the scores and overall feedback, then opens the recording or the transcript where a score looks borderline. That check takes minutes instead of an hour on a call.
How does an AI technical interview compare with technical interview outsourcing?
The 3 ways this team could have covered first technical screens. The middle column describes outsourced human interviewers in general, since providers differ.
| Question | Senior engineers screen in-house | Outsourced human interviewers | Live AI technical interview |
|---|---|---|---|
| Who runs the screen | Your own senior engineers | An outside engineer working from your brief | The AI, from the rubric your engineers wrote |
| How you pay | Engineer hours taken from the roadmap | Varies by provider; often charged per interview | Monthly plan with an interview allowance; top-ups beyond it |
| Scheduling | Match candidate and engineer calendars | Match candidate and interviewer calendars | The candidate books their own slot |
| What you get back | Notes, if someone writes them | A write-up; format varies by provider | 1 to 5 per criterion, written feedback, transcript and recording |
| Live coding | Whatever your team uses | Often included; check each provider | None; the interview is spoken |
AI Interview starts at $99/month on monthly plans you can cancel anytime. Outsourced provider pricing is not compared here.
What does this case study not show?
- One company, one 8-week hiring push, so use it to plan your own backend hiring.
- The interview is spoken, with no code editor or whiteboard; keep any hands-on exercise for the final round.
- Rubric ownership stays with your hiring managers; the criteria change only when they change them.
- The setup dates from August 2026. The product has changed since, and this page describes today's workflow.
What still needed senior engineers?
Final rounds and every hiring decision. Screening hours went to 0, but engineers still ran the final rounds, and with 71% of finalists passing, those hours were far better spent. The Cognitive also has no code editor, so a team that wants to watch someone write code should keep a hands-on exercise in the final round.
Calibration needed engineers too. The first reports were too verbose for busy reviewers, so the team cut down what they saw. And someone had to own the rubric: when the hiring managers changed what a pass meant for a role, the criteria had to change with it.
A senior engineer's week, before and after. Before: about 15 team hours of screens, each with prep and a write-up, and final rounds where 72 of every 100 candidates failed. After: no screens, a short daily read of reports, and final rounds where 71 of every 100 passed. The 15 hours went back to the roadmap the VP Engineering refused to hand to the interview calendar.
Which engineering teams should use technical interviews as a service?
Do
- Backend, platform or full-stack roles with hundreds of applicants.
- Teams where senior engineers run first screens on sprint time.
- Hiring managers who can write down what a pass looks like.
- Teams that want to hear from candidates a resume filter would drop.
Do not
- Roles where watching someone code must be the first filter.
- A single senior hire with a handful of known candidates.
- Teams unwilling to revise the rubric after the first reports.
Try it on one backend role
Write the rubric with one hiring manager, invite every applicant from this week and compare the reports with your last few human screens. Start free at https://app.thecognitive.io/signup.
Where to go next
AI recruiting for tech hiring · Hire backend developers with AI · Backend developer interview questions · Technical interview software · Pricing · All case studies
Sources and notes
Figures and quote reported by the customer in The Cognitive case study record for Series B SaaS Company, San Francisco; setup dated August 2026 (record dated 4 August 2026). Derived figures are our arithmetic and are labelled. The rubric scene and example questions are illustrations, not the customer's own. The workflow is described as the product works today.
Benchmarks, all read 5 October 2026: Ashby 2026 Talent Trends, recruiter productivity (17.6 interviews, 23.3 interview hours and about 48 days to offer per technical hire; 24.7 hours for engineering); Gem 2026 Recruiting Benchmarks (8% of applicants pass initial screening); BLS Occupational Outlook, software developers ($64.44 median hourly pay in 2025 for software developers, QA analysts and testers).
Frequently asked questions
What is technical interview as a service?
It means an outside service runs your first technical interviews so your engineers do not have to. Some services use freelance human interviewers. The Cognitive runs it as a live, two-way AI video interview scored against your rubric, and returns a report with a transcript and recording for your engineers to review.
What is technical interview outsourcing?
Technical interview outsourcing is paying an outside provider to run technical screens, usually with contract engineers who follow your brief. It adds capacity without taking your engineers' time. An AI technical interview covers the same first round from inside your own account, with every candidate scored on one written rubric.
How do you hire software engineers without burning senior engineers' time?
Take first screens off their calendars and give them reports instead. This San Francisco SaaS company moved its technical screens to The Cognitive's AI interview, cut senior screening from 15 hours a week to 0 and hired 11 engineers in 8 weeks. Engineers kept the final rounds.
What is a good final-round pass rate?
There is no universal benchmark, but a low rate means the steps before the final round are not filtering. This team's rate was 28%, so 72 of every 100 final rounds ended in a no. After every applicant took an AI technical interview first, it rose to 71%.
How many interviews does a technical hire take on average?
Ashby's 2026 Talent Trends data shows 17.6 interviews and 23.3 hours of interview time per technical hire, with engineering at 24.7 hours (read 5 October 2026). This team took senior engineers out of the first round entirely, so their hours went only on final rounds.
How long does it take to hire a backend engineer?
Ashby's 2026 benchmark for technical roles is about 48 days from application to accepted offer, read 5 October 2026. This San Francisco team was at 49 days to offer before moving its technical screens to AI interviews, and 16 days after.
What does an hour of engineer screening time cost?
At the BLS 2025 median for software developers, QA analysts and testers, $64.44 an hour (read 5 October 2026), before benefits, prep and write-up. This team's 15 weekly screening hours over 8 weeks come to 120 hours, about $7,733 of salary by our arithmetic.
What should a backend technical screen assess?
How the candidate designs, debugs and reasons about data and failure. Our suggested weights: system design 30, debugging 25, data modelling 20, reliability thinking 15, communication 10. Spoken answers show reasoning well; keep a hands-on coding exercise for the final round.
Can an AI run a system design interview?
Yes, as a spoken conversation. The Cognitive's AI asks a design question, then follows up on the candidate's answer: tradeoffs, failure handling, what they would measure. There is no whiteboard or code editor, so candidates describe their design out loud.
Do senior engineers still interview candidates?
Yes, in the final rounds. In this case study engineers stopped running first technical screens and spent that time on final rounds with candidates who had already passed the AI interview. A person still makes every hiring decision.
How long is an AI technical interview?
A recruiter sets each role to 10 or 20 minutes. For technical roles, 20 minutes gives the AI room to follow a design or debugging answer several levels deep.
Does the AI reject candidates automatically?
No. Each report suggests a verdict from the weighted score, and a hiring manager accepts or overrules it. Nothing is rejected automatically, and integrity flags such as tab switches are logged for review rather than scored.
Does it work with our ATS?
Yes, through 60+ ATS integrations, including Greenhouse, Lever, Ashby, Workday and iCIMS. Candidates imported from the ATS get a note with the score, summary and report link. Stages do not move on their own; your team moves them.
How much does an AI technical interview service cost?
AI Interview starts at $99/month on monthly plans you can cancel anytime. Each plan includes a monthly interview allowance; past it you buy an interview top-up, and a team that screens more applicants can move to a larger plan. Signup is self-serve at app.thecognitive.io/signup.
Explore: AI interviewer for backend developers · AI screening software · The cost of a manual interview · How AI interview scoring works · SaaS case study: 18 engineers · How the AI interviewer works