Releases: explodinggradients/ragas
Releases · explodinggradients/ragas
v0.3.7
What's Changed
- refactor: improve metrics code quality by @anistark in #2337
- chore: remove old analtyics by @jjmachan in #2338
- Fix/query distribution robustness by @yatoyun in #2340
- Simplify earlier how to guides in docs by @sanjeed5 in #2319
- docs: reorganize prompt evaluation guides in navigation by @sanjeed5 in #2346
- Metrics migration, migrate rouge + answer relevance by @rhlbhatnagar in #2335
- fix: streamline theme extraction from overlaps in MultiHopSpecificQue… by @kenzoyan in #2347
- Test/metric new compare by @anistark in #2349
- feat: bleu score migrated to collections by @anistark in #2352
- fix: Add List[List[str]] formats for overlapped items in theme extration (Continuation in #2347) by @kenzoyan in #2355
- feat: string metrics migrated to collections by @anistark in #2356
- feat: answer similarity migrated to collections by @anistark in #2358
- fix: add missing props token_usage_parser for test generation methods #2359 by @bhkj9999 in #2360
- feat: add bypass_n option to LangchainLLMWrapper for n-completion control by @SimFG in #2354
- docs: Add how-to guide for aligning LLM-as-Judge by @sanjeed5 in #2348
New Contributors
- @yatoyun made their first contribution in #2340
- @kenzoyan made their first contribution in #2347
- @bhkj9999 made their first contribution in #2360
- @SimFG made their first contribution in #2354
Full Changelog: v0.3.6...v0.3.7
Contributors
jjmachan, anistark, and 6 other contributors
Assets 2
v0.3.6
What's Changed
- Feature/chrf score by @kauabh in #2221
- Fix/asyncio by @anistark in #2294
- Fix: update simple RAG init to use embed_text(s) (docs) by @s3pi in #2292
- Update _bleu_score.py by @kauabh in #2297
- Refactor/update gemini to genai sdk by @sahusiddharth in #2240
- Feature/metrics input flexibility by @anistark in #2298
- Ensure old_temperature is set correctly. Fixes #1937 and #2110 by @claudepi in #2295
- Enhance EmbeddingExtractor to support both async and sync methods for… by @telesoho in #2286
- Tokens counting by @anistark in #2299
- Fix/tool call accuracy by @anistark in #2300
- fix: coroutine warning for bleu by @anistark in #2301
- Add base_url parameter to embedding_factor by @anistark in #2303
- fix: add disallowed_special on tiktoken encode by @anistark in #2304
- Feat/tool call f1 1893 by @anistark in #2305
- Feature/azure token usage extraction by @anistark in #2306
- fix: improve metric decorators with better validation and error handling by @jjmachan in #2302
- Metric/parallel tool call by @anistark in #2307
- Fix: avoid ambiguous truth value for empty numpy array in HuggingfaceEmbeddings (fixes #2080) by @Rahul2512Chauhan in #2308
- Devpod cn/main by @anistark in #2309
- Feat/quoted spans metric by @anistark in #2311
- Fix noise sensitivity compute by @anistark in #2312
- Corrected numerous typos in Markdown files. by @ker2xu in #1994
- Deprecation warnings for LLMs and Prompts by @rhlbhatnagar in #2253
- Docs/eval_rag_agent - how to evaluate and improve rag app by @sanjeed5 in #2293
- Add llamaindex agentic evals gemini by @anistark in #2317
- fix: type str in tests by @anistark in #2318
- Fix generate_multiple caching issue (#1980) by @Rahul2512Chauhan in #2314
- fix: metric inheritance patterns: separate factory-created metrics from class-instantiated metrics by @jjmachan in #2316
- fix: concurrent ResponseRelevancy by @anistark in #2328
- fix: answer_relevancy scoring logic to prevent false zero by @anistark in #2327
- feat: Add OCI Gen AI Integration for Direct LLM Support by @harshil-sanghvi in #2321
- feat: Add save/load functionality and improved repr for LLM-based metrics by @jjmachan in #2320
- Fix: Fixed the Numpy 3.13 issue by @Prigoistic in #2282
- refactor: docs and warnings for metric base new structure by @anistark in #2333
- fix: typing by @anistark in #2334
New Contributors
- @kauabh made their first contribution in #2221
- @s3pi made their first contribution in #2292
- @claudepi made their first contribution in #2295
- @telesoho made their first contribution in #2286
- @ker2xu made their first contribution in #1994
- @harshil-sanghvi made their first contribution in #2321
- @Prigoistic made their first contribution in #2282
Full Changelog: v0.3.5...v0.3.6
Contributors
jjmachan, anistark, and 11 other contributors
Assets 2
v0.3.5
What's Changed
- Docs/howto-texttosqlagent by @sanjeed5 in #2264
- fix: preview logo was too small. by @anistark in #2277
- modified the documentation to be in sync with current output format by @kotalaraghava in #2281
- removed some meta properties to test by @jjmachan in #2278
- feature: improve async / executor functionality by @ahgraber in #2070
- modification of the translate instruction by @anistark in #2284
- Remove experimental from docs and fix examples in docs by @sanjeed5 in #2270
- fix: resolve TypeError in TopicAdherenceScore bitwise operations by @anistark in #2258
- Knowledge graph/optimize for large corpus by @anistark in #2267
- Update _nv_metrics.py by @titericz in #2053
- Add telemetry by @rhlbhatnagar in #2260
- OpenAI model cost by @anistark in #2287
- docs: agent metrics code examples improvement by @yesidc in #1983
- Prompt Optimization Tutorial by @sahusiddharth in #1993
- Feature/metric type checking by @anistark in #2288
- improved the release script for
ragas-examplesby @jjmachan in #2289 - fix: removed the need for regex patterns by @jjmachan in #2290
New Contributors
- @kotalaraghava made their first contribution in #2281
- @yesidc made their first contribution in #1983
Full Changelog: v0.3.4...v0.3.5
Contributors
jjmachan, anistark, and 7 other contributors
Assets 2
v0.3.5rc2
v0.3.5rc1
What's Changed
- Docs/howto-texttosqlagent by @sanjeed5 in #2264
- fix: preview logo was too small. by @anistark in #2277
- modified the documentation to be in sync with current output format by @kotalaraghava in #2281
- removed some meta properties to test by @jjmachan in #2278
- feature: improve async / executor functionality by @ahgraber in #2070
- modification of the translate instruction by @anistark in #2284
- Remove experimental from docs and fix examples in docs by @sanjeed5 in #2270
- fix: resolve TypeError in TopicAdherenceScore bitwise operations by @anistark in #2258
- Knowledge graph/optimize for large corpus by @anistark in #2267
- Update _nv_metrics.py by @titericz in #2053
- Add telemetry by @rhlbhatnagar in #2260
- OpenAI model cost by @anistark in #2287
- docs: agent metrics code examples improvement by @yesidc in #1983
- Prompt Optimization Tutorial by @sahusiddharth in #1993
- Feature/metric type checking by @anistark in #2288
- improved the release script for
ragas-examplesby @jjmachan in #2289
New Contributors
- @kotalaraghava made their first contribution in #2281
- @yesidc made their first contribution in #1983
Full Changelog: v0.3.4...v0.3.5rc1
Contributors
jjmachan, anistark, and 7 other contributors
Assets 2
v0.3.4
What's Changed
- Update context_precision.md by @anupamck in #2262
- Reduce find_indirect_clusters() runtime through neighborhood detection and sampling by @ahgraber in #2144
- fix: handle langchain multiple batching by @anistark in #2257
- chore: fix the dead space in the header for docsite by @jjmachan in #2265
- total bin coverage for default_transform() in Knowledge Graph transformations by @tolgaerdonmez in #1950
- Improved context precision documentation by @anupamck in #2266
- Fix docs: correct inheritance class for Non-LLM metrics by @AlanPonnachan in #2272
- added rb2b analytics by @jjmachan in #2273
New Contributors
- @anupamck made their first contribution in #2262
- @tolgaerdonmez made their first contribution in #1950
- @AlanPonnachan made their first contribution in #2272
Full Changelog: v0.3.3...v0.3.4
Contributors
jjmachan, anistark, and 4 other contributors
Assets 2
v0.3.3
What's Changed
- feat: refactor Ragas
embeddingsto maintain backward compatibility by @NirantK in #2184 - feat: move
tracingfrom experimental to ragas/integrations by @NirantK in #2188 - feat: Refactor LLM structure and add InstructorLLM by @NirantK in #2191
- fix: setup and local ci by @anistark in #2199
- docs: correct small typos in Testset Generation page and related md components by @deibid in #2123
- Docs/how-to-benchmark-new-llm-guide by @sanjeed5 in #2158
- fix: access model fields via class by @emmanuel-ferdman in #2146
- add: PR template for better review and overview of changes by @anistark in #2202
- Move
promptsto ragas by @NirantK in #2197 - Fix: make the answer be the same as the split sentences by @wlbksy in #2128
- Merge workspace.toml into pyproject.toml by @NirantK in #2189
- Cleanup/move-benchmark-llm-to-examples by @sanjeed5 in #2203
- fix: shields badge for downloads replace with pepy. by @anistark in #2204
- refactor: removing some section to streamline pr template by @anistark in #2206
- Move dataset module to main package and update imports by @NirantK in #2205
- fix: type check for pyright by @anistark in #2211
- refactor: move ragas out to project root. by @anistark in #2209
- chore: fix the OIDC error in claude code by @jjmachan in #2212
- fix: _version.py is to be ignored in git by @anistark in #2213
- chore: only review for
/claude-reviewcommand by @jjmachan in #2214 - Fix a couple of typos in
docs/getstarted/evals.mdby @sadra-barikbin in #2081 - Fix repeating template bug in _nv_metrics.py by @dev-nid in #2216
- fix: ragas dir structure path fixes by @anistark in #2215
- fix: minor typo in context_precision.md by @AbhishekRP2002 in #2050
- fix: update docstrings for single-hop use case by @terry07 in #1947
- Fix issue with cache hits by @Nicba1010 in #1977
- Use generate_multiple instead of generate in a loop in ResponseRelevancy by @aabanic in #1978
- fix: update required columns in RefusalRate (docs) by @DmitrySirakov in #2008
- Add support for make install-minimal to speed up CI + dev by @rhlbhatnagar in #2218
- Move experimental metrics to main package by @NirantK in #2220
- Fix e2e test failures with datasets>=4.0.0 and missing dependencies by @anistark in #2222
- Move Utils / Exceptions module to main package and update imports by @rhlbhatnagar in #2219
- update compare_embeddings doc links by @resitaydin in #2037
- chore: add
py.typedmarker file to support type checking by @sisp in #2029 - Retire
ragas.experimentalnamespace by @NirantK in #2223 - chore: cleanup docs for ragas directory by @anistark in #2224
- fix: update make commands to use uv for docs by @anistark in #2217
- Remove experimental folder by @NirantK in #2225
- Docs/how-to-iterate-prompt by @sanjeed5 in #2226
- security: stop supporting v0.2 security releases by @anistark in #2230
- Optimize factual correctness metric runtime by 50% by @vignesh14052002 in #2153
- Fixed the mermaid which was breaking on navigation by @ganeshrvel in #2121
- Fix typos and clean style by @kwinkunks in #2042
- Feature/examples-as-installable-package by @sanjeed5 in #2210
- chore: added reo.js analytics by @jjmachan in #2232
- Move tutorials by @rhlbhatnagar in #2235
- Move howtos by @rhlbhatnagar in #2236
- Move references to docs/ by @rhlbhatnagar in #2234
- Fix BadRequestError while using OpenAI O1 series models | Add option to bypass temperature by @vignesh14052002 in #2151
- feature!: knowledge graph flexibility by @ahgraber in #2030
- return False when pred_sequence length and ref_requence length is dif… by @licux in #1894
- Feature: improve relationship builders for better async and reduced memory utilization by @ahgraber in #2077
- Mirgate all leftover docks from experimental by @rhlbhatnagar in #2243
- Add deprecation warnings for embeddings by @rhlbhatnagar in #2244
- set temperature to 0.01 for better llm compatibility by @0Falli0 in #939
- docs: fix broken link in FAQ (metrics tracing) by @Rahul2512Chauhan in #2238
- fix: temperature change to 0.01 bugs by @anistark in #2247
- Scrubbed the references to app.ragas.io by @ganeshrvel in #2207
- feat: cancellable task by @anistark in #2242
- update: add og images and metadata to docs by @anistark in #2248
- fix: add support for both langchain llm and ragas llm by @anistark in #2229
- chore: analytics support for commonroom and scarf by @jjmachan in #2250
New Contributors
- @deibid made their first contribution in #2123
- @wlbksy made their first contribution in #2128
- @sadra-barikbin made their first contribution in #2081
- @dev-nid made their first contribution in #2216
- @AbhishekRP2002 made their first contribution in #2050
- @terry07 made their first contribution in #1947
- @Nicba1010 made their first contribution in #1977
- @aabanic made their first contribution in #1978
- @rhlbhatnagar made their first contribution in #2218
- @resitaydin made their first contribution in #2037
- @sisp made their first contribution in #2029
- @vignesh14052002 made their first contribution in #2153
- @kwinkunks made their first contribution in #2042
- @0Falli0 made their first contribution in #939
- @Rahul2512Chauhan made their first contribution in #2238
Full Changelog: v0.3.2...v0.3.3
Contributors
ganeshrvel, kwinkunks, and 22 other contributors
Assets 2
v0.3.3rc1
What's Changed
- feat: refactor Ragas
embeddingsto maintain backward compatibility by @NirantK in #2184 - feat: move
tracingfrom experimental to ragas/integrations by @NirantK in #2188 - feat: Refactor LLM structure and add InstructorLLM by @NirantK in #2191
- fix: setup and local ci by @anistark in #2199
- docs: correct small typos in Testset Generation page and related md components by @deibid in #2123
- Docs/how-to-benchmark-new-llm-guide by @sanjeed5 in #2158
- fix: access model fields via class by @emmanuel-ferdman in #2146
- add: PR template for better review and overview of changes by @anistark in #2202
- Move
promptsto ragas by @NirantK in #2197 - Fix: make the answer be the same as the split sentences by @wlbksy in #2128
- Merge workspace.toml into pyproject.toml by @NirantK in #2189
- Cleanup/move-benchmark-llm-to-examples by @sanjeed5 in #2203
- fix: shields badge for downloads replace with pepy. by @anistark in #2204
- refactor: removing some section to streamline pr template by @anistark in #2206
- Move dataset module to main package and update imports by @NirantK in #2205
- fix: type check for pyright by @anistark in #2211
- refactor: move ragas out to project root. by @anistark in #2209
- chore: fix the OIDC error in claude code by @jjmachan in #2212
- fix: _version.py is to be ignored in git by @anistark in #2213
- chore: only review for
/claude-reviewcommand by @jjmachan in #2214 - Fix a couple of typos in
docs/getstarted/evals.mdby @sadra-barikbin in #2081 - Fix repeating template bug in _nv_metrics.py by @dev-nid in #2216
- fix: ragas dir structure path fixes by @anistark in #2215
- fix: minor typo in context_precision.md by @AbhishekRP2002 in #2050
- fix: update docstrings for single-hop use case by @terry07 in #1947
- Fix issue with cache hits by @Nicba1010 in #1977
- Use generate_multiple instead of generate in a loop in ResponseRelevancy by @aabanic in #1978
- fix: update required columns in RefusalRate (docs) by @DmitrySirakov in #2008
- Add support for make install-minimal to speed up CI + dev by @rhlbhatnagar in #2218
- Move experimental metrics to main package by @NirantK in #2220
- Fix e2e test failures with datasets>=4.0.0 and missing dependencies by @anistark in #2222
- Move Utils / Exceptions module to main package and update imports by @rhlbhatnagar in #2219
- update compare_embeddings doc links by @resitaydin in #2037
- chore: add
py.typedmarker file to support type checking by @sisp in #2029 - Retire
ragas.experimentalnamespace by @NirantK in #2223 - chore: cleanup docs for ragas directory by @anistark in #2224
- fix: update make commands to use uv for docs by @anistark in #2217
- Remove experimental folder by @NirantK in #2225
- Docs/how-to-iterate-prompt by @sanjeed5 in #2226
- security: stop supporting v0.2 security releases by @anistark in #2230
- Optimize factual correctness metric runtime by 50% by @vignesh14052002 in #2153
- Fixed the mermaid which was breaking on navigation by @ganeshrvel in #2121
- Fix typos and clean style by @kwinkunks in #2042
- Feature/examples-as-installable-package by @sanjeed5 in #2210
- chore: added reo.js analytics by @jjmachan in #2232
- Move tutorials by @rhlbhatnagar in #2235
- Move howtos by @rhlbhatnagar in #2236
- Move references to docs/ by @rhlbhatnagar in #2234
- Fix BadRequestError while using OpenAI O1 series models | Add option to bypass temperature by @vignesh14052002 in #2151
- feature!: knowledge graph flexibility by @ahgraber in #2030
- return False when pred_sequence length and ref_requence length is dif… by @licux in #1894
- Feature: improve relationship builders for better async and reduced memory utilization by @ahgraber in #2077
- Mirgate all leftover docks from experimental by @rhlbhatnagar in #2243
- Add deprecation warnings for embeddings by @rhlbhatnagar in #2244
- set temperature to 0.01 for better llm compatibility by @0Falli0 in #939
- docs: fix broken link in FAQ (metrics tracing) by @Rahul2512Chauhan in #2238
- fix: temperature change to 0.01 bugs by @anistark in #2247
- Scrubbed the references to app.ragas.io by @ganeshrvel in #2207
- feat: cancellable task by @anistark in #2242
- update: add og images and metadata to docs by @anistark in #2248
- fix: add support for both langchain llm and ragas llm by @anistark in #2229
- chore: analytics support for commonroom and scarf by @jjmachan in #2250
New Contributors
- @deibid made their first contribution in #2123
- @wlbksy made their first contribution in #2128
- @sadra-barikbin made their first contribution in #2081
- @dev-nid made their first contribution in #2216
- @AbhishekRP2002 made their first contribution in #2050
- @terry07 made their first contribution in #1947
- @Nicba1010 made their first contribution in #1977
- @aabanic made their first contribution in #1978
- @rhlbhatnagar made their first contribution in #2218
- @resitaydin made their first contribution in #2037
- @sisp made their first contribution in #2029
- @vignesh14052002 made their first contribution in #2153
- @kwinkunks made their first contribution in #2042
- @0Falli0 made their first contribution in #939
- @Rahul2512Chauhan made their first contribution in #2238
Full Changelog: v0.3.2...v0.3.3rc1
Contributors
ganeshrvel, kwinkunks, and 22 other contributors
Assets 2
v0.3.2
What's Changed
- feat: save and load prompt object by @jjmachan in #2152
- refactor: use basemetric as base for metric by @anistark in #2163
- add: pre-commit hook to run ci on every commit by @anistark in #2164
- Make
AnswerSimilaritya dataclass by @hutec in #2159 - docs: Add discord link to Community page in docs by @sanjeed5 in #2157
- Fixed a small typo in context_precision.md by @ImadSaddik in #2166
- Move Experimental to inside Ragas by @NirantK in #2167
- refactor: experiment function to use model as additional optional arg by @anistark in #2165
- Add python 3.13 target, refactor on pre-commit and benchmarking tests by @anistark in #2169
- fix: enum values instead of names in testset synthesizers by @lovets18 in #2137
- Cleaner, simpler pyproject.toml by @NirantK in #2173
- Move Examples to root by @NirantK in #2176
- Add
backendto ragas main by @NirantK in #2174 - Remove txt requirements by @NirantK in #2180
- Remove black by @NirantK in #2178
- feat: move
experimentfrom experimental to ragas main by @NirantK in #2175 - feat: migrate CLI from experimental to main package by @NirantK in #2183
- Reformat examples by @NirantK in #2186
- Remove simulation by @NirantK in #2187
- refactor: avoid notebook in test. run all in test-all by @anistark in #2171
- Add Claude Code GitHub Workflow by @jjmachan in #2192
- hotfix: Include LICENSE and README.md in PyPI package by @NirantK in #2195
- fix: pypi requirements and image absolute path by @anistark in #2196
- chore: fix release script by @jjmachan in #2198
New Contributors
- @anistark made their first contribution in #2163
- @hutec made their first contribution in #2159
- @ImadSaddik made their first contribution in #2166
- @lovets18 made their first contribution in #2137
Full Changelog: v0.3.1...v0.3.2
Contributors
NirantK, jjmachan, and 5 other contributors