12 Interviewer Biases, Where AI Adds Bias, and How to Audit

By Sparsh Goyal, Founder at The Cognitive. Published 2026-04-16 · Last updated 2026-10-05

• AI bias in hiring is a tool selecting or scoring one group at a lower rate for reasons unrelated to the job; interviewer bias is the human version, and both shrink when every candidate is judged against one written standard a person reviews.
• Structured interviews predict performance at .42 versus .19 for unstructured ones (Sackett et al., 2022), and the documented AI failures came from training data, proxies, hard-coded rules and what the tool measured: Amazon, iTutorGroup, HireVue's facial analysis and the Workday case.
• Audit any tool with impact ratios by group at every stage, and check NYC Local Law 144, Colorado SB26-189 (1 January 2027), the EU AI Act (2 December 2027) and the Illinois AI Video Interview Act.
• I run The Cognitive, which uses one rubric per role, rejects nobody automatically and has not published a bias audit; outside facts were read on 4 October 2026.

AI bias in hiring is a pattern where an automated tool selects, scores or ranks one group of candidates at a lower rate for reasons unrelated to the job, usually because of its training data, proxy variables or how it measures answers. Interviewer bias is the human version of the same problem. Both are measurable, and both shrink when every candidate is judged against one written standard that a person reviews.

The research on structure is clear. In the 2022 re-analysis by Paul Sackett and colleagues in the Journal of Applied Psychology, structured interviews came out as the strongest predictor of job performance they reviewed, at a validity of .42, while unstructured interviews fell to .19.

Full disclosure: I run The Cognitive, which runs live AI interviews. We have not published a bias audit, and this guide says exactly what we do and do not do.

For the arithmetic of adverse impact, read our adverse impact explainer. For the controls inside our product, read bias and fairness at The Cognitive. This page is the practical guide between them.

What is AI bias in hiring, and what is interviewer bias?

AI bias in hiring is a systematic difference in how an automated tool treats candidates from different groups, when the difference has nothing to do with the work. It can come from a resume screener, a ranking model, a chatbot that asks knockout questions or a scoring step after an interview.

Interviewer bias is a systematic error in how a person evaluates a candidate. The interviewer forms a view from something irrelevant, such as a shared university or a nervous first minute, and then scores the whole interview through it.

US law treats the 2 alike. The federal Uniform Guidelines define a "selection procedure" as any measure used to make an employment decision, and they name informal interviews in that definition. A casual phone screen counts in the same way an algorithm does.

The legal test for both is adverse impact: does one group get selected at a substantially lower rate? Swapping a human screen for an AI screen does not remove your duty to check outcomes. It changes where bias can enter and how easy it is to measure.

What are the most common types of interview bias?

These are the interviewer biases I see most often in debriefs, with a structural fix for each. A structural fix works without asking the interviewer to try harder, because good intentions do not survive a busy afternoon.

The name row has some of the strongest evidence. In a field experiment published by the National Bureau of Economic Research in 2003, Marianne Bertrand and Sendhil Mullainathan sent matched resumes to real job ads, and white-sounding names received 50 percent more callbacks than African-American-sounding names.

Time of day belongs on the list as well. A recruiter running the 6th screen of a Friday listens differently from the 1st on Monday. I have no clean hiring study to cite for it, so treat it as a scheduling rule: cap screens per person per day.

Attention bias: when the interviewer stops listening

Attention bias happens when an interviewer fails to listen fully, then reads the gaps in their own attention as gaps in the candidate. The candidate explains how they rolled back a failed migration while an incident thread scrolls past on the interviewer's second monitor. The feedback says "couldn't go deep".

It hides inside normal hiring language. "Lacked depth", "too theoretical" and "low energy" sound like judgment, and none of them names the answer that failed. Quiet candidates and non-native speakers lose the most, because the interviewer has to work harder to follow them.

You can find it with 10 recent recordings, rejects first. Watch the interviewer as closely as the candidate and mark each one for:

• Phone check: the interviewer looks down at a phone.
• Competing apps: Slack, email or unrelated tabs on screen.
• Missed answer: a question the candidate already answered gets asked again.
• Missed question: the candidate asks something the interviewer does not hear.
• Weak follow-up: an answer that needed a probe gets "got it".
• Evidence gap: the feedback makes claims with no example.
• Mismatch: the recording shows a stronger candidate than the feedback describes.

Then set a floor every human interviewer meets: one screen, phone out of reach, camera on, rubric open, notes written during each answer. A simple interview notes template makes the last part a habit.

Fear bias: when nobody wants to own the yes

Fear bias is rejecting a qualified candidate because approving them feels personally risky. A bad hire becomes a story with names attached, while a great candidate you turned down disappears into another company and nobody counts the cost. So "no" feels safe. It shows up in 4 patterns:

• The vague veto. One person says the candidate "lacks depth" and cannot point to the answer.
• The moving bar. A new criterion appears in the final round that was never in the rubric.
• The committee shrug. Everyone is a private yes and nobody is a public one.
• The comparison trap. "Let's see who else is out there" keeps the role open another month.

Fear bias feeds on decisions made from memory and feeling. Agree the criteria before anyone interviews, let no concern decide the outcome unless someone can show the answer behind it, and put the cost of "no" in every debrief: days open, people interviewed, work waiting.

Are structured interviews less biased than unstructured ones?

Structured interviews predict job performance about twice as well as unstructured ones, according to the most recent large re-analysis. Sackett, Zhang, Berry and Lievens argued in the Journal of Applied Psychology in 2022 that earlier estimates had over-corrected for range restriction, and they recalculated.

Structured interviews landed at .42, the top of the procedures they reviewed. Unstructured interviews dropped from the .38 reported by Schmidt and Hunter in 1998 to .19.

Validity measures how well a method predicts performance, so it is a separate number from fairness. The link to bias is practical.

An unstructured interview lets each interviewer choose what to ask, what to weigh and what to remember, and each choice is a door for the biases in the table. A structured interview closes most of those doors before the call starts. It has 4 parts:

1. Criteria written before the first interview, tied to the job, with weights.
2. Anchored rating scales that describe what a 1, a 3 and a 5 answer sound like.
3. Every criterion probed for every candidate. The wording can change with the conversation; the standard does not.
4. Independent scoring before the debrief, so the loudest person does not set everyone's number.

Structure alone does not cure attention or fear bias: a distracted interviewer can ask the right question and miss the answer.

How do you reduce bias in interviews?

Reduce bias in interviews by deciding the standard before you meet anyone and by making every decision show its evidence. On top of the 4 parts of structure above, add these 4 habits:

1. Replace "culture fit" with the behaviour you need, such as "explains trade-offs to non-engineers".
2. No evidence, no rejection. "Not technical enough" becomes "which question exposed the gap?" and "poor communication" becomes "which answer was unclear, and did we ask for clarification?"
3. Keep the final decision with a named person, and record why.
4. Measure outcomes by stage with the four-fifths check below, at least once a year. Our diversity recruiting answer covers widening the pool before the first screen.

Is AI hiring biased?

AI hiring can be biased, and the documented cases show how. It can also be less biased than the process it replaces, if it applies one written standard to everyone and keeps a record you can check. The difference comes down to what the tool reads, what it learned from and whether anyone measures its outcomes.

Where AI adds bias

Training data. A model trained on past decisions learns their patterns. Reuters reported in October 2018 that Amazon's experimental recruiting engine had been trained on 10 years of resumes, most of them from men. It penalised resumes that included the word "women's", as in "women's chess club captain", and downgraded graduates of 2 all-women's colleges.

Amazon edited it to be neutral to those terms, with no guarantee the model would not find other ways to sort people, and disbanded the team. Recruiters looked at its recommendations but never relied solely on them, Reuters reported.

Proxy variables. A tool that never sees age, race or disability can still learn them from something that tracks them. Zip code can stand in for race, graduation year for age, and employment gaps or medical leave for disability or caregiving.

Hard-coded rules. Some of the clearest cases involve no machine learning at all. In September 2023 iTutorGroup agreed to pay $365,000 to settle an EEOC lawsuit.

The EEOC said its software was programmed to automatically reject female applicants aged 55 or older and male applicants aged 60 or older, turning away more than 200 qualified US applicants. A knockout question in an ATS can do the same.

Evaluation method. What a tool measures can disadvantage a group even when the inputs look neutral. HireVue stopped using facial analysis in March 2020, SHRM reported in February 2021, after the company found visual analysis "no longer significantly added value".

Scoring tone, accent or facial movement risks measuring background instead of ability. The same question applies to tests: our guide to personality tests for hiring covers what they predict and where they fall short.

Where the Workday case stands

Mobley v. Workday is the case buyers ask about most. In May 2025 Judge Rita Lin of the Northern District of California granted preliminary certification of an age discrimination collective: applicants aged 40 and over who applied through Workday's platform since 24 September 2020 and were denied employment recommendations.

In June 2026 the court dismissed some amended claims and let others proceed, including California FEHA claims and disability claims built on proxies such as employment gaps. As of 4 October 2026 I found no ruling on the merits. The case tests whether a software vendor, not only the employer, can be liable for a screening tool's outcomes.

Where AI reduces bias

The strongest case for AI is consistency. A human panel gives candidates different interviewers on different days.

An AI interview can hold every candidate for a role to one rubric, probe every criterion, and keep a transcript and recording a reviewer can check against the score (more on that in our piece on interview transcripts). That reduces bias when the rubric is job-related, the scoring reads answers instead of appearance, a person decides, and someone measures outcomes.

How do you audit an AI hiring tool for bias?

Audit an AI hiring tool by comparing selection rates across groups at every stage it touches, then checking what it reads and who reviewed it. The checklist:

1. Map the decision points: each step where the tool selects, scores, ranks or tags people.
2. List the inputs the model reads at each point. Every input is a possible proxy.
3. Find any automatic rejection, where a candidate can leave the process unseen by a person.
4. Collect self-reported demographics through your voluntary EEO process, kept apart from the decision. NYC's rules do not allow inferred or imputed demographics in a bias audit.
5. Calculate selection rates and impact ratios by sex, race and ethnicity, and their intersections, at each stage.
6. Read the result with care, using the 3 cautions below.
7. Repeat yearly and whenever the tool, the rubric or the role mix changes.

The four-fifths rule, worked through

This example is illustrative. The numbers are invented to show the arithmetic and come from no real employer or tool.

Say an AI interview step decides who moves to a hiring manager. 300 men and 200 women complete it. 120 men advance, a selection rate of 40%. 60 women advance, a selection rate of 30%. Divide the lower rate by the highest: 30 divided by 40 is 0.75. The federal four-fifths rule treats a ratio below 0.80 as evidence of adverse impact, so this step needs investigating.

The 3 cautions:

• Small numbers. The EEOC's guidance says the rule is unreliable when numbers are so small that selecting one different person would flip the result.
• No safe harbour. A ratio above 0.80 does not clear you; the EEOC says the test "does not resolve the ultimate question of unlawful discrimination".
• The whole process. A fair interview can sit behind a biased resume screen, so measure the whole process as well as each step.

Our adverse impact explainer covers the formula and the small-sample problem in depth.

Which AI hiring tools audit for bias, and what should you ask vendors?

Under NYC Local Law 144 the bias audit is the employer's job, done by an independent auditor. Vendors can commission one and several publish summaries, but the city's FAQ says the employer stays responsible for making sure an audit was done. Put these 6 questions to any vendor, us included, and keep the answers on file:

1. Has an independent auditor reviewed the tool? Who, when, on what data, and can I read the summary with its impact ratios?
2. What does the model read when it scores: resume, transcript, video, audio, location?
3. Was it trained or tuned on past hiring decisions, and whose?
4. Can any candidate be rejected without a person seeing them?
5. Who writes the criteria, and can my team edit them for each role?
6. Can I export results by stage to run my own four-fifths check?

If a vendor answers "our AI is unbiased" with no audit, no input list and no export, treat the claim as unsupported.

What do the laws on AI hiring bias require?

Here are the main laws as of 4 October 2026, with dates and links. This is a summary, not legal advice; ask your employment counsel how each applies to your roles.

US federal law: Title VII and the four-fifths rule

Title VII bans employment practices with a disparate impact on a protected group unless the employer shows they are job-related and consistent with business necessity. The Uniform Guidelines supply the four-fifths rule and tell employers to keep records of each selection procedure's impact. Both apply to AI tools as they do to people.

NYC Local Law 144: bias audits for automated employment decision tools

Local Law 144 took effect on 1 January 2023, with enforcement from 5 July 2023. An employer may not use an automated employment decision tool to screen candidates for New York City jobs unless it has had an independent bias audit within the past year, reporting impact ratios by sex, by race and ethnicity, and by their intersections.

The employer must post a summary on the employment section of its website and notify candidates at least 10 business days before use.

That answers the search for "NYC Local Law 144 compliant AI hiring tools". No tool is compliant by itself, because the duties fall on the employer that uses it. The law also covers assessing people who applied; the city's FAQ puts sourcing and outreach out of scope. Our AI screening software compliance guide lists 10 laws side by side, and for sourcing that widens who you reach, see our picks of diversity recruiting software.

Enforcement has been light. A New York State Comptroller audit published on 2 December 2025 found the city's complaint handling ineffective. The city reviewed 32 companies and found 1 instance of non-compliance; the Comptroller found at least 17 potential instances in the same group.

Colorado SB26-189: automated decision-making from 1 January 2027

Colorado's governor signed SB26-189 on 14 May 2026. It repeals and replaces the 2024 Colorado AI Act before that law took effect, and its requirements start on 1 January 2027. An employer using automated decision-making technology that materially influences a hiring decision must:

• give clear notice at the point of interaction;
• after an adverse outcome, explain in plain language, within 30 days, how the system influenced the decision;
• let the worker request meaningful human review;
• keep records for 3 years.

EU AI Act: high-risk obligations from 2 December 2027

The EU AI Act lists AI used for recruitment or selection as high-risk in Annex III, including tools that filter applications and evaluate candidates. High-risk tools are allowed, with obligations attached.

The Digital Omnibus on AI, approved by the Council of the EU on 29 June 2026, moved the date those obligations apply to stand-alone Annex III systems from 2 August 2026 to 2 December 2027. Our EU AI Act page covers what that means for recruiting teams.

Illinois Artificial Intelligence Video Interview Act

Since 1 January 2020, an employer that uses AI to analyse recorded video interviews for Illinois-based roles must tell applicants beforehand, explain how the AI works and what general characteristics it uses, and get consent.

Videos may be shared only with people needed to evaluate the applicant and must be deleted within 30 days of a request. Since 2022, an employer that relies solely on AI analysis to choose who gets an in-person interview must report race and ethnicity data to the state each year.

What does The Cognitive do about bias, and what has it not done?

The Cognitive is AI recruiting software that sources candidates across ~900M public profiles and interviews them in a live, two-way AI video interview. This is what it does structurally, checked against our code on 4 October 2026:

• One rubric per role. Each role has fixed, weighted criteria, and each criterion describes what a strong 5 out of 5 answer looks like. The AI decides questions live, so the wording changes while the standard holds.
• Scores come from the transcript. The evaluator scores each criterion from what the candidate said and is instructed to ground every observation in it.
• The suggested verdict follows a fixed rule. It is derived from the weighted score out of 100, so the same score always gets the same verdict.
• Nothing is rejected automatically. A person reads the report and decides. If scoring fails, the interview goes to manual review.
• Integrity flags are logged, not scored. The reviewer sees a flag; it does not lower the score. Our guide to AI interview cheating detection explains what those signals mean.
• The resume check is told to ignore pedigree. Its instruction says not to reward brand prestige, formatting or length, and not to penalise employment gaps, non-traditional backgrounds or writing style.
• No demographic filters. Sourcing has no filter for age, gender, race or any other protected trait.
• Candidates know it is AI. The invitation tells candidates the interview is run by AI.

Now the limits:

• We have not published a bias audit, and the app does not calculate impact ratios for you.
• Telling a model to ignore prestige is a design choice we have not measured.
• The evaluator reads the job description and the candidate's resume alongside the transcript, so it is not blind to background.

If you use our interview score to decide who advances for a New York City role, that use may count as an automated employment decision tool, and you would need your own independent audit first.

Each report gives a 1 to 5 score per criterion plus overall written feedback, a weighted score out of 100, a suggested verdict, and the transcript and recording, so a reviewer can check a score against what was said.

This 5-minute recording, made on 1 September 2026, shows a live AI interview for a go-to-market role. The AI asks the candidate about an outbound email campaign, then follows up on reply rates and on how leads were qualified, building each question on the previous answer.

AI Interview starts at $99/month and AI Sourcing at $49/month. Our AI interviewer page and AI interviewer guide explain how the interview works.

Hold every candidate to one written rubric Live, two-way AI interviews scored per criterion from the transcript, with a person making every decision. Start free

How do you start this week?

1. Pick one open role.
2. Write its 5 or 6 criteria with anchors before the next interview.
3. Pull 10 recent recordings and run the attention audit.
4. Adopt "no evidence, no rejection" at the next debrief.
5. Run a four-fifths check on last quarter's screen-to-interview stage.

To try a fixed rubric and a live AI interview on a real role, The Cognitive has a free trial with 100 sourcing credits.

Run your next screen against one written standard Describe the role, set the rubric, and read a scored report for every candidate you interview. Start free

Sources

• Sackett, Zhang, Berry and Lievens (2022), Journal of Applied Psychology 107, 2040-2068, as summarised by SIOP's TIP: siop.org, read 4 October 2026
• Revised validity table, structured and unstructured interviews versus Schmidt and Hunter (1998): master-hr.com, read 4 October 2026
• Bertrand and Mullainathan, NBER Working Paper 9873 (July 2003): nber.org, read 4 October 2026
• EEOC, iTutorGroup to pay $365,000 (11 September 2023): eeoc.gov, read 4 October 2026
• Reuters (Jeffrey Dastin), Amazon scraps secret AI recruiting tool that showed bias against women (10 October 2018), via CNBC: cnbc.com, read 4 October 2026
• SHRM, HireVue discontinues facial analysis screening (3 February 2021): shrm.org, read 4 October 2026
• CDF Labor Law, preliminary certification in Mobley v. Workday (16 May 2025 order): cdflaborlaw.com, read 4 October 2026
• Duane Morris, motion to dismiss ruling in Mobley v. Workday (25 June 2026): blogs.duanemorris.com, read 4 October 2026
• Uniform Guidelines, 29 CFR 1607.4 and 1607.16: law.cornell.edu, read 20 September 2026
• EEOC, Questions and Answers on the Uniform Guidelines: eeoc.gov, read 20 September 2026
• NYC DCWP, Automated Employment Decision Tools FAQ: nyc.gov, read 20 September 2026
• New York State Comptroller, Enforcement of Local Law 144 (2 December 2025): osc.ny.gov, read 4 October 2026
• Colorado General Assembly, SB26-189 Automated Decision-Making Technology: leg.colorado.gov, read 4 October 2026
• ArentFox Schiff, Colorado replaces its landmark AI Act with SB 26-189 (22 May 2026): afslaw.com, read 4 October 2026
• EU AI Act, Annex III: artificialintelligenceact.eu, read 20 September 2026
• Council of the EU, artificial intelligence timeline: consilium.europa.eu, and Cuatrecasas, Council of the EU approves Digital Omnibus on AI: cuatrecasas.com, read 4 October 2026
• Illinois Artificial Intelligence Video Interview Act, 820 ILCS 42/5 to 42/20: ilga.gov, read 4 October 2026

All posts · thecognitive.io