Treat paywall status as retrieval quality.

Treat paywall status as retrieval quality. A system that ranks paywalled pages as top evidence will score well in offline benchmarks that only measure relevance and then fail when users click. Kagi’s new setting to hide paywalled links makes the point: entitlement constraints belong in the retrieval model and in your evaluation harness. If you run RAG or browser-based agents, add an access check and penalize inaccessible sources in your scorer, and include citation reachability in your tests before you trust the leaderboard.