Skip to content

My First RAG Bot

Difficulty: Intermediate
Time: 45 minutes
Learning Focus: Document indexing, vector retrieval, context-based generation
Module: RAG

Overview

Learn how to use the rag CLI to index documents and create a simple Retrieval-Augmented Generation (RAG) system that can answer questions based on your own notes or documents.

Instructions

Step 1: Set up your environment

  1. Install the rag CLI tool: 
    pip install hands-on-ai
  2. Ensure you have access to the testdata/demo_notes/ directory or prepare your own documents

Step 2: Index your documents

  1. Browse through the available demo documents: 
    ls hands_on_ai/testdata/demo_notes/
  2. Create an index of all documents in the demo notes directory: 
    rag index hands_on_ai/testdata/demo_notes/
  3. Observe the output showing how many files were found and processed, and note the name of the saved index file (e.g., sample_index.npz)

Step 3: Ask questions using your RAG system

  1. Ask a basic question about the content: 
    rag ask "What does TCP do?" --show-context
  2. Try different questions to explore the content: 
    rag ask "How does HTTP work?" --show-context
rag ask "What is the difference between TCP and UDP?" --show-scores
  3. Experiment with different flags:
  4. --show-context: See what information was used to answer
  5. --show-scores: View relevance ranking of different text chunks
  6. --top-k 5: Retrieve 5 most relevant chunks instead of default 3

Step 4: Understand how RAG works

  1. Review how the system processes your request:
  2. Documents are broken into smaller chunks
  3. Vector embeddings are created for each chunk
  4. When you ask a question, it finds chunks with similar embeddings
  5. Retrieved chunks are used as context for generating an answer

Extension Ideas

  1. Create your own knowledge base:
  2. Create markdown files with your class notes or research

  3. Index them and ask questions to test your understanding

  4. Experiment with different file types:

  5. Add PDF files to your document collection
  6. Re-index to include the new files

  7. Compare how well the system handles different formats

  8. Build a specialized RAG system:

  9. Create a collection of documents on a specific topic
  10. Develop a set of test questions to evaluate retrieval quality

  11. Fine-tune your prompts to get better answers

  12. Analyze retrieval performance:

  13. Use the --show-scores flag to see how different chunks are ranked
  14. Experiment with question phrasing to improve retrieval accuracy
  15. Try adjusting the number of chunks retrieved with --top-k