v0.2.1

What's Changed

• Fix Discord link by @NirantK in #1498
• Fix Quickstart link by @NirantK in #1499
• Rewrite Open Analytics by @NirantK in #1500
• Suggestions for Introduction and Key Features by @NirantK in #1501
• docs: fix broken links in testset generation custimisation by @jjmachan in #1502
• Update Bedrock modules for LangChain v0.3.x by @minorun365 in #1487
• Fixes: Quickstart Ragas Metrics by @NirantK in #1504
• Add Contributors by @NirantK in #1505
• docs: Change test_size parameter name to testset_size by @Youngrok123 in #1507
• docs: readme add llm by @shahules786 in #1511
• docs: add faq by @shahules786 in #1503
• fix: propagate callbacks in RagasOutputParser by @dlyz in #1513
• docs: fix usage of eval dataset by @shahules786 in #1514
• feat: use ensure_ascii=False for KnowledgeGraph files by @dlyz in #1518
• fix: import error for TestsetGeneration and small fixes by @jjmachan in #1516

New Contributors

• @NirantK made their first contribution in #1498
• @minorun365 made their first contribution in #1487
• @dlyz made their first contribution in #1513

Full Changelog: v0.2.0...v0.2.1

v0.2.0

What's Changed

• feat: topic adherence metric by @shahules786 in #1327
• feat: tool call accuracy by @shahules786 in #1311
• feat: sql sementic equivalence by @shahules786 in #1323
• feat: Factual correctnes metric by @shahules786 in #1334
• docs: migrating over to mkdocs by @jjmachan in #1301
• feat: Adding transforms for testset generation by @jjmachan in #1344
• docs: traditional metrics by @shahules786 in #1346
• feat: transforms for Knowledge Graphs by @jjmachan in #1345
• docs: core concepts in metrics by @shahules786 in #1347
• chores: cleanup metrics by @shahules786 in #1348
• docs: refactor core concepts by @shahules786 in #1351
• wrap as_complete with async by @liupgd in #1350
• chores: add datasets to init by @shahules786 in #1356
• docs: new getting started by @shahules786 in #1355
• docs: metrics concepts by @shahules786 in #1361
• docs: add mindmap by @shahules786 in #1362
• Fixed Nav, search, and some styling issues for the docs by @joelk9895 in #1363
• feat: transform engine to execute transforms by @jjmachan in #1352
• chores: remove fuzzywuzzy by @shahules786 in #1369
• docs: component guides by @shahules786 in #1365
• chores: fix typo by @shahules786 in #1370
• fix:ensure correct sequence in tool calls by @shahules786 in #1371
• [howtos] update Langfuse cookbook by @jannikmaierhoefer in #1368
• feat: adding an implementation of abstractQA by @jjmachan in #1359
• feat: implement TestsetGenerator by @jjmachan in #1375
• docs: testset generation by @shahules786 in #1373
• feat: added more simulators by @jjmachan in #1376
• Update models in Ragas documentation: change gpt-3.5-turbo-16k to gpt-4o-mini and gpt-4 to gpt-4o by @suekou in #1380
• fix: aspect_critique documentation by @jjmachan in #1382
• docs: added google analytics to mkdocs by @jjmachan in #1384
• docs: fix index for metric/index.md by @jjmachan in #1388
• fix: fixed verdict comparison in evolutions.py by @MarkHmnv in #1366
• feat: output new Testset object for generation by @jjmachan in #1386
• Update answer_relevance.md: Minor spell correct by @soumendrak in #1390
• feat: support for callbacks and traces in testset generation by @jjmachan in #1392
• non-asii characters support in faithfulness by @gargewp in #1396
• feat: get and set prompts mixin by @shahules786 in #1391
• fixes: rename naming in testset generation by @shahules786 in #1393
• chore!: moving to pydantic2 by @jjmachan in #1394
• docs: improve getting started by @shahules786 in #1398
• Fix typo in metric error message by @suekou in #1404
• feat!: moving experimental.testset to replace old testset by @jjmachan in #1401
• feat: renamed sample operations and cleaned up test warnings by @jjmachan in #1407
• added better styling to reder jupyter notebooks nicely. by @zodwick in #1389
• docs: add core concepts for test generation by @shahules786 in #1402
• docs:Auto Gen of Reference Page by @zodwick in #1408
• fix key error for FaithfulnesswithHHEM by @linzzzzzz in #1410
• fix: remove usage of contexts in response relevancy by @shahules786 in #1409
• fix: score for jupyter by @shahules786 in #1411
• feat: NER extractor and Jaccard by @shahules786 in #1406
• feat(styles): refactor CSS to use custom variables by @zodwick in #1413
• docs : add customisation by @shahules786 in #1412
• feat(docs): add support for reusable snippets by @zodwick in #1414
• Feature/css variables by @zodwick in #1415
• docs: fix nav bar by @shahules786 in #1416
• feat: more features to prompt object by @jjmachan in #1418
• chore: add tracking for new testset generation by @jjmachan in #1419
• feat: adapt prompt to different languages by @jjmachan in #1429
• chores: add reference to gitignore by @shahules786 in #1428
• docs: redo intro by @shahules786 in #1427
• Add Custom Ragas Light and Dark Themes (v1) by @zodwick in #1431
• fix attribute error for faithfulness metric by @linzzzzzz in #1436
• docs: language adaptation by @shahules786 in #1446
• fix: data export by @shahules786 in #1461
• feat: save and load Prompts by @jjmachan in #1458
• refactor: convert faithfulness, answer_correctness and noise_sensitivity to new prompt object by @jjmachan in #1462
• Minor typo in _answer_relevance.py by @pjanowski in #1452
• feat: added repr for prompt objects by @jjmachan in #1459
• feat: change context recall and precision prompts to new by @jjmachan in #1467
• docs: fix invalid links and imports by @shahules786 in #1473
• feat: changed summary to new prompt by @jjmachan in #1469
• docs: redo readme by @shahules786 in #1480
• docs: new build process with rtd and API documentation by @jjmachan in #1474
• chore: added install for mkdocs insiders by @jjmachan in #1481
• chores: update gitignore by @shahules786 in #1483
• docs: improve testgen namings by @shahules786 in #1489
• Fixed an issue where non-ASCII characters were changed to Unicode characters within the prompt by @Youngrok123 in #1490
• docs: fix search by @shahules786 in #1491
• feat: added save and load to RagasDataset by @jjmachan in #1492
• docs: moved from ipnb files to md files by @jjmachan in #1482
• feat: set ensure_ascii to false by @jjmachan in #1494
• feat: small refactors and cleanups by @jjmachan in #1493

New Contributors

• @liupgd made their first contribution in #1350
• @joelk9895 made their first contribution in #1363
• @jannikmaierhoefer made their first contribution in #1368
• @suekou made their first contribution in #1380
• @MarkHmnv made their first contribution in #1366
• @soumendrak made their first contribution in #1390
• @gargewp made their first contribution in #1396
• @zodwick made their first contribution in #1389
• @linzzzzzz made their first contribution in https://git...

v0.2.0-beta

What's Changed

• feat: topic adherence metric by @shahules786 in #1327
• feat: tool call accuracy by @shahules786 in #1311
• feat: sql sementic equivalence by @shahules786 in #1323
• feat: Factual correctnes metric by @shahules786 in #1334
• docs: migrating over to mkdocs by @jjmachan in #1301
• feat: Adding transforms for testset generation by @jjmachan in #1344
• docs: traditional metrics by @shahules786 in #1346
• feat: transforms for Knowledge Graphs by @jjmachan in #1345
• docs: core concepts in metrics by @shahules786 in #1347
• chores: cleanup metrics by @shahules786 in #1348
• docs: refactor core concepts by @shahules786 in #1351
• wrap as_complete with async by @liupgd in #1350
• chores: add datasets to init by @shahules786 in #1356
• docs: new getting started by @shahules786 in #1355
• docs: metrics concepts by @shahules786 in #1361
• docs: add mindmap by @shahules786 in #1362
• Fixed Nav, search, and some styling issues for the docs by @joelk9895 in #1363
• feat: transform engine to execute transforms by @jjmachan in #1352
• chores: remove fuzzywuzzy by @shahules786 in #1369
• docs: component guides by @shahules786 in #1365
• chores: fix typo by @shahules786 in #1370
• fix:ensure correct sequence in tool calls by @shahules786 in #1371
• [howtos] update Langfuse cookbook by @jannikmaierhoefer in #1368
• feat: adding an implementation of abstractQA by @jjmachan in #1359
• feat: implement TestsetGenerator by @jjmachan in #1375
• docs: testset generation by @shahules786 in #1373
• feat: added more simulators by @jjmachan in #1376
• Update models in Ragas documentation: change gpt-3.5-turbo-16k to gpt-4o-mini and gpt-4 to gpt-4o by @suekou in #1380
• fix: aspect_critique documentation by @jjmachan in #1382
• docs: added google analytics to mkdocs by @jjmachan in #1384
• docs: fix index for metric/index.md by @jjmachan in #1388
• fix: fixed verdict comparison in evolutions.py by @MarkHmnv in #1366
• feat: output new Testset object for generation by @jjmachan in #1386
• Update answer_relevance.md: Minor spell correct by @soumendrak in #1390
• feat: support for callbacks and traces in testset generation by @jjmachan in #1392
• non-asii characters support in faithfulness by @gargewp in #1396
• feat: get and set prompts mixin by @shahules786 in #1391
• fixes: rename naming in testset generation by @shahules786 in #1393
• chore!: moving to pydantic2 by @jjmachan in #1394
• docs: improve getting started by @shahules786 in #1398
• Fix typo in metric error message by @suekou in #1404
• feat!: moving experimental.testset to replace old testset by @jjmachan in #1401
• feat: renamed sample operations and cleaned up test warnings by @jjmachan in #1407
• added better styling to reder jupyter notebooks nicely. by @zodwick in #1389
• docs: add core concepts for test generation by @shahules786 in #1402
• docs:Auto Gen of Reference Page by @zodwick in #1408
• fix key error for FaithfulnesswithHHEM by @linzzzzzz in #1410
• fix: remove usage of contexts in response relevancy by @shahules786 in #1409
• fix: score for jupyter by @shahules786 in #1411
• feat: NER extractor and Jaccard by @shahules786 in #1406
• feat(styles): refactor CSS to use custom variables by @zodwick in #1413
• docs : add customisation by @shahules786 in #1412
• feat(docs): add support for reusable snippets by @zodwick in #1414
• Feature/css variables by @zodwick in #1415
• docs: fix nav bar by @shahules786 in #1416
• feat: more features to prompt object by @jjmachan in #1418
• chore: add tracking for new testset generation by @jjmachan in #1419

New Contributors

• @liupgd made their first contribution in #1350
• @joelk9895 made their first contribution in #1363
• @jannikmaierhoefer made their first contribution in #1368
• @suekou made their first contribution in #1380
• @MarkHmnv made their first contribution in #1366
• @soumendrak made their first contribution in #1390
• @gargewp made their first contribution in #1396
• @zodwick made their first contribution in #1389
• @linzzzzzz made their first contribution in #1410

Full Changelog: v0.1.20...v0.2.0-beta

v0.1.21

Full Changelog: v0.1.16...v0.1.21

• Pins to langchain<0.3
• based on v0.1.16 which was the last working version for 0.1
• hopefully last release before v0.2

v0.1.20

What's Changed

• fix: pin langchain_core to <3 by @jjmachan in #1329

Full Changelog: v0.1.19...v0.1.20

v0.1.19

What's Changed

• feat: add non llm based context recall by @shahules786 in #1266
• feat: Non LLM based context precision by @shahules786 in #1264
• feat: added n and temperature passthrough to prompt by @jjmachan in #1281
• Add option to disable progress bar in evaluate by @freddiev4 in #1279
• feat: new aspect critic metrics by @shahules786 in #1286
• feat: add simple scorer metric by @shahules786 in #1291
• docs: add docs for new metrics by @shahules786 in #1292
• Fix wrong input key name by @ruanhao in #1300
• chore: move rubrics based metric into single file by @shahules786 in #1287
• feat: Instance specifc rubrics metrics by @shahules786 in #1304
• feat: agent goal accuracy metric by @shahules786 in #1303
• docs: agent metrics by @shahules786 in #1314
• Update Opik notebook example and integration by @jverre in #1295
• fix: column rename by @shahules786 in #1324

New Contributors

• @freddiev4 made their first contribution in #1279
• @ruanhao made their first contribution in #1300

Full Changelog: v0.1.18...v0.1.19

v0.1.18

What's Changed

• fix: inverse distance by @shahules786 in #1267
• fix: non llm based metrics by @shahules786 in #1268
• fix: v1 to v2 dataset by @shahules786 in #1275
• fix: make score nested if loop_is_running by @jjmachan in #1276

Full Changelog: v0.1.17...v0.1.18

v0.1.17

What's Changed

• update documentation for faithfulness with HHEM by @Miaoranmmm in #1242
• New data representation by @shahules786 in #1222
• Feat: Opik Integration by @jverre in #1256
• Add support to MultiTurn Metrics by @shahules786 in #1249
• fix: add single_turn_ascore by @shahules786 in #1261
• fix: handle v0.1 dataset by @shahules786 in #1262
• Non LLM based metrics by @shahules786 in #1260
• feat: set and get prompts for metrics by @shahules786 in #1259

New Contributors

• @jverre made their first contribution in #1256

Full Changelog: v0.1.16...v0.1.17

v0.1.16

What's Changed

• load HHEM on specified device by @Miaoranmmm in #1235
• feat: new Prompt object and faithfulness metric by @jjmachan in #1232
• feat (experimental) added new prompt and metric into ragas.experimental by @jjmachan in #1240

New Contributors

• @Miaoranmmm made their first contribution in #1235

Full Changelog: v0.1.15...v0.1.16

v0.1.15

What's Changed

• fix: typo in model name by @shahules786 in #1198
• metrics: add domain specific rubrics based scoring by @vaishakhRaveendran in #1189
• docs: rubrics based metrics by @shahules786 in #1200
• docs: add context utilisation to the index by @shahules786 in #1202
• Docs: fix typo in metrics by @shahules786 in #1203
• Implementation of Noise sensitivity metrics from RAGChecker by @sahusiddharth in #1190
• Tiny typo fix: it's -> its by @peterkaminski in #1219
• fix : solves #1215 - Inconsistency in the Value of EVALMODE_TO_COLUMNS[EvaluationMode.ca] by @jucgonzalezes in #1216
• chore: removed extra of requirements by @jjmachan in #1225
• Bugfix of evaluate function in llama_index.py by @vcim in #1223
• [FIX] - Fix for summarization edge case by @sky-2002 in #1201

New Contributors

• @peterkaminski made their first contribution in #1219
• @jucgonzalezes made their first contribution in #1216
• @vcim made their first contribution in #1223

Full Changelog: v0.1.14...v0.1.15