GAIA v0.10.1 Release Notes

🌟 TL;DR - Key Highlights

• 📄 Document Q&A Evaluation - Process PDF documents and evaluate AI models on question-answering tasks
• 🛠️ Installation Fixes - Resolved critical installer issues with shortcuts and CLI launching
• 🔧 Build Infrastructure - Added GitHub workflows for future app distribution
• 🔄 Updated Dependencies - Latest RAUX (v0.2.4) and Lemonade (v8.1.5) for improved stability

🚀 New Features

Document Q&A Evaluation Support

Enhanced the evaluation framework with comprehensive PDF processing capabilities for question-answering experiments.

• PDF text extraction using the pypdf library for local document processing
• Dedicated Q&A configuration (basic_qa.json) for document-based evaluation experiments
• Enhanced groundtruth consolidation for Q&A pairs and RAG use cases
• Document content caching for improved performance during batch experiments

💡 Quick Start:

# Generate Q&A groundtruth from PDFs
gaia groundtruth -d ./data/pdf --use-case qa --num-samples 3 -o ./groundtruth

# Run Q&A experiments
gaia batch-experiment -c ./src/gaia/eval/configs/basic_qa.json -i ./groundtruth/consolidated_qa_groundtruth.json -o ./experiments

# Evaluate and visualize results
gaia eval -d ./experiments -o ./evaluation
gaia visualize --experiments-dir ./experiments --evaluations-dir ./evaluation

GitHub Workflows for App Distribution

Added foundational infrastructure for building and packaging GAIA mini-apps:

• PR validation workflows for app builds
• Release build automation
• Automated packaging workflows

🐛 Bug Fixes

• Fix gaia visualize webapp not found error
• Installer improvements - Fixed NSIS installer flow for proper shortcut creation after RAUX installation
• CLI launching - Resolved issues with launch_gaia.bat --cli execution
• Package distribution - Added missing package extras ([talk,dev,eval,youtube,audio]) to installer
• CI/CD enhancements - Improved release script to exclude test workflows from distribution

🔄 Updates

• RAUX Integration updated to version 0.2.4 (v0.6.5+raux.0.2.4) fixing installation failures that showed blank screens instead of error messages in the terminal display
• Lemonade Backend updated to version 8.1.5 for improved performance and stability
• Package structure - Updated setup.py with complete module structure (chat, talk, apps, eval, mcp, agents)
• Added Discord label

📖 Documentation

The Document Q&A evaluation workflow is documented in detail at
https://github.com/aigdat/gaia/blob/main/docs/eval.md#workflow-3-document-qa.