Evaluate an applicant tracking system
Run the same five ATS demo tasks with each supplier, test role permissions and exports, and record what the recruiter could complete unaided.
Give every shortlisted supplier a copy of the same fictional vacancy, candidate records and instructions. Ask for a test account on the plan being quoted. Let the recruiter drive the keyboard while a colleague records completion time, assistance and anything the supplier needs to configure. Use made-up contact details and a test mailbox controlled by your team.
Download the blank ATS demo record (.txt).
Give each vendor the same five tasks
| Task | What the recruiter does | What the observer checks |
|---|---|---|
| Open a vacancy | Enter the approved brief and prepare a job-board advert | Salary band and location survive the preview. A draft stays private |
| Resolve a duplicate | Add a second application from one fictional person for a different role | Both applications keep their vacancy IDs and separate histories |
| Withdraw after booking | Arrange an interview, then withdraw the candidate | Interview and reminders are cancelled. The reason and action are recorded |
| Restrict a hiring manager | Sign in as a manager who can see only one vacancy | Searching, opening a direct link and exporting all respect the same access limit |
| Exit the system | Export vacancies, applications, notes and sample attachments | The buyer can connect each record to its vacancy and open the files outside the platform |
Define “complete” before the demo. If a real job board cannot be tested in the trial, ask for a sandbox or a dated example and record that dependency. Put a follow-up and owner against any unanswered question. Use a separate test tenant or disabled mail delivery to keep invitations and withdrawals within the evaluation team.
Record what happened during the test
Merebridge Recruitment, our fictional agency, tests the withdrawal on two suppliers. In the first system, the recruiter's screen changes immediately, but a queued reminder still arrives in the controlled mailbox. In the second, the test user withdraws the applicant and the scheduled message disappears. The observer waits through the scheduled send time before recording the result.
This changes the shortlist. The first supplier can remain under consideration if it demonstrates the corrected behaviour on the quoted plan before the decision date. A promised development date belongs in the follow-up column.
Check the work after the demo
Ask which actions need a paid administrator, which job boards incur charges and whether someone can retrieve attachments when the subscription ends. Try opening an exported file yourself. Check that names with apostrophes, long vacancy references and candidates with two applications have survived intact.
Agree what applicant information your team needs and who should review, retain or remove it. Ask the supplier to demonstrate how an authorised user applies that policy and how exceptions are recorded. Put essential permissions or export requirements in the pass conditions before adding the marks to the supplier scorecard.