AI Search Arena

I Search Arena is an independent monthly benchmark platform that evaluates 27+ AI search optimization (GEO/AEO) tools using a rigorous, transparent methodology. Built to be the definitive reference for practitioners choosing between tools in a rapidly evolving market.

Each benchmark cycle scores every tool across 51 dimensions — spanning AI visibility, content optimization, technical implementation, and more — using 6 independent AI models (OpenAI, Anthropic, Google, Cohere, Mistral, Meta) via a consensus methodology. Scores are synthesized using median aggregation with confidence tags derived from inter-model agreement, ensuring no single model's bias influences results.

The platform features a full operator console for managing benchmark cycles through a 10-state lifecycle from draft and planning through evaluation, synthesis, vendor review, and publication. Tools are enrolled per cycle, evaluated in parallel across all models, and ranked using weighted composite scores with dense ranking.

Transparency is core to the design: the complete scoring methodology is published, every cycle produces a SHA-256 sealed audit package, and vendors receive a 5-business-day review window to submit factual corrections before publication. Market segment filters allow users to compare tools within their specific context — enterprise SEO, e-commerce, agency, SMB, and more.

Built as a solo founder with an AI-assisted development workflow, shipping the full platform from database schema to production in under a week.

Practitioners evaluating AI search optimization tools had no independent, data-driven benchmark to compare options. Vendor marketing claims were unverifiable, tool comparisons were subjective blog posts, and no standardized scoring methodology existed. Teams were making expensive tool decisions based on anecdote rather than evidence.

Designed a consensus-based evaluation system where 6 independent AI models score each tool across 51 dimensions, then synthesize results using median aggregation to eliminate individual model bias. Built a 10-state cycle lifecycle with guards and side effects to enforce data integrity at every step — tools can't advance to evaluation without minimum enrollment, scores can't publish without sealed audit packages. Vendor review windows and full methodology publication ensure accountability. The entire platform ships as a single Next.js deployment with Prisma, Neon PostgreSQL, and OpenRouter as the AI gateway.

• Consensus beats individual assessment — Using 6 models with median synthesis and confidence tags based on inter-model agreement produced far more reliable scores than any single model could. Standard deviation directly maps to confidence: low disagreement = high confidence.
• State machines enforce process integrity — The 10-state cycle lifecycle with transition guards (e.g., minimum 5 tools per track before evaluation) prevented every "shortcut" scenario that would have compromised data quality.
• Transparency is a feature, not overhead — SHA-256 audit packages, published methodology, and vendor review windows initially felt like extra work but became the platform's core differentiator and credibility driver.
• Design for resumability — The evaluation pipeline makes ~10,000 API calls per cycle. Making every stage idempotent and resumable meant network failures and timeouts were non-events rather than cycle-killing problems.