---
title: RAG Pipeline Optimizer
emoji: 🔍
colorFrom: blue
colorTo: purple
sdk: streamlit
sdk_version: 1.28.0
app_file: app/dashboard.py
pinned: false
license: mit
---

# 🔍 RAG Pipeline Optimizer

A comprehensive evaluation platform for comparing Retrieval-Augmented Generation (RAG) pipelines with multiple configurations.

## 🌟 Features

- **6 Pipeline Configurations**: Compare different chunking, embedding, and retrieval strategies
- **Modular Architecture**: Pluggable components (document loaders, chunkers, embedders, retrievers, generators)
- **Interactive Dashboard**: Built with Streamlit for real-time testing and analysis
- **Comprehensive Evaluation**: LLM-as-Judge framework with multiple metrics
- **Cost Analysis**: Track API costs across different pipeline configurations
- **Batch Testing**: Evaluate multiple queries simultaneously
- **Performance Leaderboard**: Compare pipeline performance side-by-side

## 🚀 Quick Start

### Using the Space

1. Select a pipeline configuration from the sidebar
2. Enter your query in the text input
3. View results with retrieved documents and generated responses
4. Compare different pipelines in the Comparison tab
5. Run batch evaluations with predefined test queries

### Pipeline Configurations

- **Pipeline A**: Small chunks (256) + Fast embeddings
- **Pipeline B**: Medium chunks (512) + Quality embeddings  
- **Pipeline C**: Large chunks (1024) + Hybrid retrieval
- **Pipeline D**: Semantic chunking + Reranking
- **Pipeline E**: Sentence-window retrieval
- **Pipeline F**: Custom optimized configuration

## 🏗️ Architecture

```
rag_optimizer/
├── app/                    # Streamlit dashboard
│   ├── pages/             # Multi-page app components
│   └── dashboard.py       # Main app entry
├── core/                  # RAG pipeline components
│   ├── chunker.py        # Text chunking strategies
│   ├── embedder.py       # Embedding models
│   ├── retriever.py      # Vector search & retrieval
│   ├── generator.py      # LLM response generation
│   └── evaluator.py      # Pipeline evaluation
├── config/               # Pipeline configurations
└── scripts/              # Evaluation & analysis tools
```

## 📊 Evaluation Metrics

- **Retrieval Accuracy**: Precision, Recall, F1 Score
- **Response Quality**: LLM-as-Judge scoring
- **Latency**: End-to-end query time
- **Cost**: API costs per query
- **Context Relevance**: Retrieved document quality

## 🛠️ Technology Stack

- **Vector Database**: ChromaDB
- **Embeddings**: sentence-transformers (MiniLM, MPNet, etc.)
- **LLMs**: OpenAI GPT, Anthropic Claude, Groq (via API)
- **Framework**: Streamlit
- **Datasets**: Wikipedia, Natural Questions

## 🔑 API Keys

To use all features, add your API keys in the Space settings:

- `OPENAI_API_KEY` - For GPT models
- `ANTHROPIC_API_KEY` - For Claude models  
- `GROQ_API_KEY` - For free Groq inference

## 📈 Use Cases

- **RAG System Design**: Compare configurations before production deployment
- **Hyperparameter Tuning**: Optimize chunk size, top-k, and other parameters
- **Model Selection**: Evaluate different embedding models and LLMs
- **Cost Optimization**: Find the best quality/cost trade-off
- **Research**: Benchmark RAG approaches on standard datasets

## 🎯 Performance Insights

The platform helps you answer:
- Which chunk size works best for your use case?
- Is reranking worth the extra latency?
- Which embedding model gives best retrieval accuracy?
- What's the optimal top-k for your domain?
- How do different LLMs compare on your queries?

## 📝 Example Queries

Try these sample queries to test the system:

- "What is the capital of France?"
- "Explain quantum entanglement"
- "Who won the 2023 Nobel Prize in Physics?"
- "What are the main features of Python programming language?"

## 🤝 Contributing

This is a personal research project. Feel free to fork and adapt for your own use cases!

## 📄 License

MIT License - see LICENSE file for details

## 👨‍💻 Author

**Pujan Patel**  
- LinkedIn: [linkedin.com/in/pujan-patel](https://linkedin.com/in/pujan-patel)
- Email: pujan3p@gmail.com
- GitHub: [github.com/pujan3p](https://github.com/puji4ml)

## 🔗 Links

- [Project Documentation](https://github.com/puji4ml/RAG-Pipeline-Optimizer---MLOps)

---