The Cognitive
The 7-Day Sourcing Agent Test
Every AI sourcing demo looks perfect. The vendor picked the role. Here you pick it.
Under 5Walk awayof the first 20 your manager would call
5 to 9Extendone more role, one more week
10+Buyif the proof holds too
The 7 days
Each day has one job and one pass mark.
DAY 1Pick 3 real rolesDecide what you're testing before any vendor sees it
| Role | Pick | What it proves |
|---|---|---|
| A | A role you filled last year | You know the right answer. Does it find that person, or people like them? |
| B | An open role, normal difficulty | What a normal week looks like |
| C | Your hardest open role | Whether it copes with a thin market |
- Name one hiring manager per role. They grade on Days 3 and 5.
- Write down each role's hard limits now: place, level, work authorisation.
- Tell no vendor which roles you picked until Day 3.
Done when: 3 roles, 3 graders, and the hard limits are written down.
DAY 2One brief for every toolSame words, no vendor help, one refine each
test-brief.md
Role: [title] Place: [city + commute limit, or remote + time zones] Level: [who they report to, who reports to them] Must-haves (max 3, written as work done): 1. [verb + thing + size. "Closed $60K deals with CFOs"] 2. 3. Never: [manager's hard no] Skip: [current clients, people already in your ATS]
- For Role A, write the brief as the job looked on the day you opened it. Leave out the hire's name and employer.
- Paste the same text into every tool.
- Vendors don't touch the brief, and no one on their team runs the search for you.
- Each tool gets one refine, used on Day 5.
Done when: the same 3 briefs are saved, word for word.
DAY 3Count fits in the first 20The first 20 as ranked. No scrolling for better ones
- Run each brief once. Export the first 20 people in the order shown.
- Remove the tool's name and scores, and mix the lists.
- Send them to the hiring manager with the note below.
- Count only Call.
- Role A: can you find your actual hire, or someone with the same work behind them?
- Count anyone who breaks a hard limit from Day 1.
note-to-hiring-manager
Testing a sourcing tool on [role]. Here are 20 profiles. Mark each one in 30 seconds: Call = I'd speak to them this week Maybe = only if we run out of people No = wrong person Tell me the reason for any No in three words.
Pass: 10+ of 20 Call on average · hire or close match found · 0 or 1 hard limits broken
DAY 4Check the proofAsk why for each person, then go and look
Pick 10 people the tool ranked highest. Open each real profile next to the tool's reason.
| Counts as proof | Doesn't |
|---|---|
| "Runs payroll for 600 staff across 3 states at her current company" | "Strong skills match" |
| "Sold $50K to $140K software deals to CFOs since 2022" | "94% fit" |
| "Moved a monolith to services, listed under his 2021 job" | "Great background in the space" |
- Can you find the claim on the profile?
- Is it about their work, or only the job title?
- Is it about the current job, or a job from years ago?
Pass: 8 of 10 reasons hold on all three questions
DAY 5The hiring manager testOne correction, then see if the tool learned
- Take the most common No reason from Day 3.
- Give each tool that one correction, using its one refine.
- Grade the next 20 people the same way, without the tool's name.
- Ask the manager the question below.
question-for-hiring-manager
You've now seen 2 rounds of profiles for [role]. If you could only interview 3 people this week, who are they? Would you trust the next list without seeing the first one?
Pass: Call count goes up after the correction, and the manager picks 3 without asking for more
DAY 6Cost per fitPrice per seat tells you nothing. Price per Call does
cost-per-fit
Tool cost for the test (seats, credits, contacts): $____ Recruiter hours on the test x hourly cost: $____ Manager hours x hourly cost: $____ Total: $____ Total / all Calls across every list = cost per fit
- Get real contact details for 5 people marked Call in each tool.
- Count how many reach a real inbox or phone. Bounces are a cost.
- Compare cost per fit with what a fit costs you today, using your last 3 hires for the same kind of role.
Pass: cost per fit beats what you pay today, with at least 3 of 5 contacts working
DAY 7Buy, extend or walk awayFill in the scorecard, then ask the vendor 4 questions
- Enter each tool's numbers in the scorecard below.
- Ask the vendor the 4 questions.
- Send the boss summary.
questions-for-the-vendor
1. Where do your profiles come from, and how old are they? 2. What happens to people's data when we delete a role or leave? 3. What do I pay for people who turn out to be a poor fit? 4. Can we keep the role, shortlist and history if we stop paying?
Done when: one decision per tool, one page to your boss
Scorecard
Type over the numbers. The verdict updates as you go.
Example numbers · replace them with your own
| Check | Pass | Tool A | Tool B | Tool C |
|---|---|---|---|---|
| Call, of first 20 (average) | 10+ | |||
| Found the hire on Role A | Yes or close | |||
| Hard limits broken, of 20 | 0 or 1 | |||
| Reasons that hold, of 10 | 8+ | |||
| Calls went up after correction | Yes | |||
| Contacts that work, of 5 | 3+ | |||
| Verdict |
Buy: passes every check. Walk away: under 5 Calls, under 5 reasons that hold, or more than 2 hard limits broken. Anything else: extend.
Boss summary
boss-summary
We tested [number] AI sourcing tools for 7 days on 3 of our own roles. Hiring managers graded every list without seeing the tool's name. Winner: [tool] - [X] of the first 20 were people our managers would call - It found [our hire / a close match] on a role we'd already filled - [X] of 10 reasons held up against the real profile - Cost per fit: $[X], against $[X] today Recommendation: [buy / extend / walk away] First role to run: [role] Decision needed by: [date]
Run it on The Cognitive
Same roles, same marks. If we score under 5 of 20, walk away from us too.
- Paste your Day 2 brief into the search box as written.
- Every person comes with reasons taken from their own profile, so Day 4 is quick to check.
- Shortlist the people your manager marks Call, then run the correction on Day 5 as a refine.
- Your trial lasts 7 days, so it fits the test.
- The trial has 100 credits, and each person shown costs 1. Three roles at 20 people each use 60. Run Day 5 on one role for 20 more, and you have 20 left for contacts.