How to verify software supplier promises before you buy

To verify a software supplier’s promise, turn it into a specific requirement, ask for evidence that applies to the product you would buy, and test the behaviour where you can. Record the result separately from the supplier’s statement. A demonstration, a written commitment and an observed test result answer different questions; keep all three distinct when deciding whether to proceed.

For a UK business, this need not mean another lengthy procurement questionnaire. Start with the few promises whose failure would stop a key task, expose important information or make switching supplier difficult. Give each one a short verification record and a named person responsible for resolving gaps.

1. Replace broad claims with a precise requirement

Preserve the supplier’s original wording, including where it appeared and the date. Then write down what your team needs it to mean. “Easy reporting”, for example, is too broad to accept or reject: identify the report, the person who needs it and the output they must obtain.

Before requesting evidence, specify:

• Scope: the product, subscription tier, modules and integrations you intend to purchase.
• Task: what a named user role must be able to do.
• Conditions: the relevant settings, permissions and data.
• Acceptance: the observable result that would satisfy your requirement.

Ask the supplier to confirm that your interpretation is included in its offer. If it depends on an extra module, configuration work or a future release, record that dependency before treating the promise as covered.

This creates a useful distinction between a purchasing requirement and a sales description. Your requirement supplies the acceptance criteria; the supplier’s response tells you whether it is offering to meet them.

2. Match the evidence to the promise

Choose evidence that answers the particular question. For a feature, request relevant documentation and a demonstration using your scenario. For an ongoing service commitment, request the proposed service terms and an explanation of how performance will be reported. For a security-process claim, request an explanation of the process and supporting assurance material.

The NCSC’s guidance for software customers is a relevant starting point for assessing suppliers’ software-security practices. Use it to structure security questions rather than relying on a general assurance that a product is “secure”.

When a supplier provides a certificate, assessment or report, ask:

• Which organisation, product and service does it cover?
• When was the assessment performed, and what period does it address?
• What was excluded from its scope?
• Are there findings or dependencies relevant to your intended use?
• What can the supplier share if the full report is confidential?

A document’s title alone cannot answer those questions. Keep its scope and date beside the claim it supports, so a colleague can see why you considered it relevant.

The government’s Software Security Code of Practice is voluntary and sets out expectations for software security and resilience. A link to the code is not evidence that a particular vendor meets those expectations. Ask a supplier referring to it to identify the practices it follows and the evidence behind its answer.

3. Run a bounded acceptance test

For behaviour you can observe, agree a small test with the supplier. Use an authorised trial or demonstration environment and fictitious data. Record the configuration and user permissions so the result is interpretable. Do not extend a feature trial into intrusive security testing without explicit agreement on authorisation and scope.

Suppose a supplier promises that your team can export its records. Your requirement is that an operations user can export a selected set, including agreed custom fields, into a file your team can open and reconcile.

A practical test would be:

1. Create ten fictitious records with known values, including a blank field, punctuation and a custom-field value.
2. Sign in with the permissions intended for the operations user.
3. Export the selected records using the proposed workflow.
4. Compare the exported record count and agreed field values against the originals.
5. Record missing values, manual steps and any administrator or supplier assistance required.

Success supports that particular export workflow under those conditions. If your requirement also includes attachments or relationships between records, include them explicitly in the test. Do not infer that they work from a successful export of simpler data.

4. Keep a decision record, not just a folder of documents

Use one entry per important promise. A shared document or spreadsheet is enough if your colleagues can find the evidence and understand the decision.

Suggested verification record

• Claim and origin: exact wording, document or email, and date.
• Required scope: product, tier, configuration and intended user.
• Acceptance criteria: the result your business needs.
• Evidence: document reference, scope, date and relevant passage.
• Test: environment, steps, expected result and observed result.
• Status: verified within scope, partly verified, unverified or failed.
• Next decision: proceed, request clarification, retest or reject for this requirement.
• Owner and review trigger: who resolves the issue, and what change requires another check.

Treat “unverified” as an evidence gap, rather than automatically as a failed feature. For a must-have requirement, resolve that gap before purchase or record who accepts the remaining uncertainty and why. If a test fails, retain the original result and retest after the proposed fix rather than replacing it with a verbal assurance.

Finally, compare the proposed order and service terms with this record. Ask the supplier to resolve differences in scope, dependencies or commitments before signing. Your colleague should be able to see both what the supplier has promised and the narrower, precise account of what your team has verified.