Discussion on EvaluationHarness

[TuanaCelik]

This is the discussion board for EvaluationHarness, an experimental component which allows you to orchestrate evaluation for a pipeline with various parameters and for a number of metrics.

The experimental feature includes the RAGEvaluationHarness designed for standard RAG pipelines.

📚 Full Documentation

🧑‍🍳 Cookbook here

Let us know what you think, and if you have any questions about this feature.

Note: Experimental features live in this repository for a fixed period of time. We don't guarantee that we will continue maintaining experimental features. But if they are successful, stable and if you like it, we will move the feature to the core Haystack package.

PS: The first version of this component was implemented by @shadeMe 🚀

[bilgeyucel]

📢 More about EvaluationHarness

Check out our latest cookbook that showcases how to effectively use EvaluationHarness with its override and comparison features: Evaluating AI with Haystack