What is a prompt testing rig?
A prompt testing rig is the DIY form of citation tracking: a written protocol that makes engine answers reproducible enough to compare over time. It has four components — the question set, the session protocol, the capture protocol and the scoring sheet. None requires software; all require discipline. It operationalizes the standard described in measuring AI visibility and the measurement standard.
How do you design the fixed question set?
Build 50-100 questions across five intent classes, weighted toward how your buyers actually decide:
- Identity (about 10%): who is X, what does X do — tests entity accuracy.
- Category (about 30%): best tools for Y, top providers of Z — tests Visibility against competitors.
- Comparison (about 25%): X vs alternatives, X vs named competitor — tests positioning language.
- Reputation (about 20%): is X reliable, X reviews, complaints about X — tests Brand Perception.
- Boundary (about 15%): pricing, integrations, edge cases — tests Semantic Coverage of specific facts.
Write each question exactly as a buyer would phrase it, freeze the wording, and never edit the set mid-cycle. If questions must change, version the set and note the break in comparability.
What does the session and capture protocol look like?
- Use fresh sessions — logged-out or fresh-profile state — so personalization history does not contaminate answers.
- Run each question once per engine per cycle; note the date, engine, mode (for engines with search toggles, test both modes and log which produced the answer — see DeepSeek’s two modes).
- Screenshot every full answer and archive it with a naming convention: date, engine, question ID. Save cited source lists too.
- Score each answer on the four dimensions, 0-100 each: Visibility (are you present?), Citation Rate (is your content cited as a source?), Semantic Coverage (how much of the correct fact set appears?), Brand Perception (is the characterization accurate and favorable?).
Why monthly, and what do you do with the output?
Monthly is the cadence that balances engine volatility against measurement labor; weekly adds noise-chasing, quarterly misses drift. After each run, compare against the previous cycle and the original baseline, and write a short delta memo: what moved, on which engine, hypothesized cause, action. When a logged engine change occurs — see the 2026 update log — run an extra cycle within two weeks. This rhythm is exactly the baseline → intervention → re-measurement loop of the 90-day POC.
When should you graduate from DIY to tooling?
Graduate when the labor, not the methodology, becomes the constraint: typically when you monitor more than four engines, manage multiple brands, or need weekly cadence. At that point evaluate platforms against the criteria in citation tracking tools — keeping your rig as the audit trail that keeps any platform honest.
How do you keep the rig honest over time?
Three rules. First, freeze the question set between cycles; change it only deliberately, with a version note. Second, rotate who runs it occasionally — a second operator catches drift in how sessions are set up and how scores are applied. Third, archive everything: the screenshots, the score sheets, the delta memos. The archive is what lets you answer, a year in, whether visibility actually improved or the questions quietly got easier.
Common pitfalls
Editing questions mid-cycle destroys the baseline. Testing in your daily logged-in browser measures your personalization bubble, not the market. Skipping screenshots leaves nothing to audit. And scoring only “were we mentioned” misses the three dimensions — citation, coverage, perception — where actionable signal lives.
Where to go next
Pair the rig with crawler log analysis, plan tool adoption with free vs paid stack, and get a fast diagnostic from the GEO Checker.