vendor-evaluation git:20260718.7c1285cA

Immutable. This exact content is served forever at /api/v1/blob/84e9dd6fc8d5c75a.

---
name: vendor-evaluation
description: Evaluate vendors against weighted criteria, a proof of concept that tests your real workload, and a clear-eyed accounting of exit costs before you sign. Use when choosing a paid tool or service you will depend on and a wrong pick is expensive to undo.
---

# Vendor evaluation

Choosing a vendor is choosing a dependency you will live with for years and pay
to leave. The evaluation goes wrong when it is run on demos and vibes: the
polished sales walkthrough decides it, criteria get invented to justify the
favorite, and no one asks what it costs to get back out. A disciplined evaluation
tests the vendor on your work and prices the exit before it prices the entry.

## Method

1. **Set weighted criteria before you see a single demo.** List what matters:
   functionality, reliability, security posture, support, total cost, and how hard
   it is to leave. Assign weights that sum to a fixed total, and write them down
   first. Criteria invented after the demo just rationalize the vendor who demoed
   best.
2. **Score against requirements, not against each other.** Rate every vendor on
   each criterion against your bar, so a weak field cannot make a mediocre option
   look strong by comparison. Separate must-haves, which are pass or fail, from
   nice-to-haves that earn weighted points.
3. **Design the proof of concept around your real workload.** Feed it your actual
   data shapes, your peak volume, your awkward edge cases, and your integration
   points. A POC on the vendor's sample data proves their demo works. Define the
   success metrics and the time box before you start so the POC ends in a verdict,
   not an open-ended trial.
4. **Cost the exit, not just the entry.** Estimate what it takes to migrate off:
   data export formats, proprietary interfaces you would rewrite, retraining, and
   contract lock-in. High switching cost is a real price even if it never appears
   on an invoice. Prefer vendors whose data you can get back in a usable form.
5. **Verify security and compliance with evidence, not the questionnaire alone.**
   Ask for the SOC 2 or ISO report, the data processing terms, subprocessor list,
   breach history, and where data is stored. A vendor who cannot produce these for
   a serious deal is telling you something.
6. **Reference-check past the list they gave you.** Talk to the references, then
   find a customer they did not hand you and ask what breaks at month six: support
   response times, hidden costs, the feature that was "on the roadmap." Sales sells
   the roadmap; a real customer tells you the product.

## Checks

- Were the weighted criteria fixed before demos, or shaped to fit a favorite?
- Did the POC run on your workload and data, or on the vendor's canned scenario?
- Can you state, in dollars and weeks, what leaving this vendor would cost?

## Boundaries

This picks the vendor; it does not negotiate the contract terms or run procurement
and legal review, which have their own owners and thresholds. Approval limits and
required security sign-offs are company policy: route spend and data agreements
through the process your organization requires rather than deciding them here.