RAG with ZenML
RAG is a sensible way to get started with LLMs.
Retrieval-Augmented Generation (RAG) is a powerful technique that combines the strengths of retrieval-based and generation-based models. In this guide, we'll explore how to set up RAG pipelines with ZenML, including data ingestion, index store management, and tracking RAG-associated artifacts.
The first part of this guide to RAG pipelines with ZenML is about understanding the basic components and how they work together. We'll cover the following topics:
why RAG exists and what problem it solves
how to ingest and preprocess data that we'll use in our RAG pipeline
how to leverage embeddings to represent our data; this will be the basis for our retrieval mechanism
how to store these embeddings in a vector database
how to track RAG-associated artifacts with ZenML
At the end, we'll bring it all together and show all the components working together to perform basic RAG inference.
Last updated