Introduction

Welcome to the Automated Evaluation Framework for Retrieval-Augmented Generation Systems (ARES). ARES is a groundbreaking framework for evaluating Retrieval-Augmented Generation (RAG) models. The automated process combines synthetic data generation with fine-tuned classifiers to efficiently assess context relevance, answer faithfulness, and answer relevance, minimizing the need for extensive human annotations. ARES employs synthetic query generation and prediction-powered inference (PPI), providing accurate evaluations with statistical confidence.

Get Started

🚀 Quick Start

Set up and try out ARES efficiently with our quick start guide!

💪 Synthetic Generation

Discover how to automatically create synthetic datasets that closely mimic real-world scenarios for robust RAG testing.

📊 Training Classifier

Learn how to train high-precision classifiers to determine the relevance and faithfulness of RAG outputs

⚙️ RAG Evaluation

Configure RAG model evaluation with ARES to accurately evaluate your model's performance.