The instrument-stability question. A finding that is true in March and false in May — with no announcement — is not a finding; it is a dated observation. Before the observatory publishes anything about what evaluators recommend, it must characterize how stable those recommendations are. This question gates the credibility of every future measurement, including our own commercial offerings.
- Affected buyers
- Anyone relying on an AI evaluator's recommendation as if it were a stable fact
- Affected categories
- All categories surfaced in AI-assisted comparison and screening
- Potential product impact
- If recommendations are noisy, single-shot "AI visibility" measurements are largely meaningless
- Current confidence
- Unmeasured by us. Instrument instability is a known risk (see FOUNDATIONAL_UNDERSTANDING, research risks).
- Evidence tier
- Narrated (untiered type)