What to validate before a paid pilot

Why the study comes before the pilot

Most disappointing AI-camera projects fail before the software is switched on: the camera could not see the condition, the lighting changed at night, nobody owned the response, or the “accuracy” was a number nobody could reproduce on site. A paid Site & Scenario Readiness Study exists to find those failures on paper, for a fraction of a pilot’s cost, and to end with a written go / no-go that belongs to you.

Whether the scenario is a warehouse dock, a mall service counter or a factory station, the same ten items decide readiness.

The ten-point site-readiness check

Score each item 0 (missing), 1 (partial) or 2 (confirmed). This is the same scoring used inside the study.

1. Visible problem. The condition you care about is actually visible from an existing camera angle, and it occupies enough pixels and time to judge.
2. Suitable cameras. Resolution, field of view, lighting across the shift, occlusion and retention are known for each candidate camera.
3. Stream and network access. RTSP/IP streams are reachable, the network segmentation is understood, and someone has authority to configure it.
4. Named workflow owner. A specific person receives the exception and closes the loop — including at 9 p.m. on a Sunday for a retail portfolio.
5. Economic buyer. The person who funds a production rollout is identified and reachable.
6. Measurable baseline. Today’s manual check is described: who does it, how often, what a miss costs. Without a baseline there is nothing to measure the pilot against.
7. Privacy path. Model A (media stays on site) or Model B (approved frames leave under documented approval) is chosen per site, and biometric processing is out of scope.
8. Support fit. Service hours and escalation are agreed in writing rather than assumed.
9. Paid-pilot budget. The organisation can fund a bounded pilot and a production path if the pilot passes.
10. Expansion potential. There is a second site or a second scenario worth reaching if the first proves out.

16–20: propose the paid study and pilot. 11–15: close the gaps first. 0–10: defer — a pilot now would consume goodwill without producing evidence.

Acceptance criteria are written before the pilot starts

A pilot is accepted against criteria agreed in advance and signed by a named owner on the customer’s side. The seven categories:

Category	What gets written down
Scenario definition	Scenario name, camera views, the operational question, the rule in plain language.
Ground truth	How many labelled test cases, who labels them, and the video window used.
Performance measures	Precision, recall, false-alert count and miss count at the agreed threshold.
Operational measure	Share of exceptions acknowledged within the agreed window; reports delivered on schedule.
Capacity measure	Throughput against the sizing estimate; latency measured end to end for your scenario and network.
Business measure	Baseline versus pilot: hours saved, missed checks reduced, incident evidence produced.
Acceptance decision	Pass, fail-with-remediation, or no-go — signed by a named owner.

Accuracy is measured during your pilot against agreed test cases. That number is real, and it is yours — which is exactly why it is not quoted in advance.

Three scenario examples

• Warehouse dock monitoring. Rule: “bay 3 attended within 10 minutes of a trailer docking”. Validate: the dock camera sees both the trailer and the bay floor; night lighting; who at the 3PL closes the exception.
• Mall operations. Rule: “information counter attended during opening hours”. Validate: counter is visible without occlusion from signage; the regional operations lead receives the weekly branch comparison.
• Factory station. Rule: “station 4 attended during operating hours”. Validate: angle shows the operator position; shift schedule loaded; production supervisor owns the exception.

What a good go / no-go contains

Camera census with suitability notes per camera; N/L/A sizing (recorded, live, continuous-AI counts); the workflow baseline; the privacy map with the Model A/B recommendation; the draft acceptance criteria; and an investment estimate. If the verdict is no-go, you keep the conclusions — that is what the study fee buys.