Why Do We Need Specialized Vector Databases? In the first five articles, we figured out how to chunk documents and generate embeddings. Now where do these vectors live, and how are they efficiently retrieved? You might wonder: "Can't I just store vectors in Redis or PostgreSQL?" No — traditional databases are designed for exact queries (e.g., WHERE id = 123), while vector retrieval is Approximat
In this article, we’ll explore the basics of Vector Databases (Vector DBs) and why they are the backbone of modern AI. What is a Vector Database? Popular Vector Databases Qdrant: Known for being high-performance and written in Rust. FAISS: Developed by Meta, optimized for efficient similarity searches. Milvus: Built for scalability and massive datasets. For this guide, we’ll focus on Qdrant becaus
In Day-1, we understood about the overview of a RAG system and what are its components and how it helps the LLM to generate more accurate and contextual responses. Now, lets see about the storage of the data using Vector Databases. Lets assume that we have a PDF with us and this would be considered as our private data. Now I want my LLM to have the context about this PDF, So that I could ask any q
Power BI is a powerful business analytics service developed by Microsoft that empowers users to visualise data and share interactive dashboards across their organisation. While Power BI can handle data from various sources, its true potential is unleashed when connected to robust data sources like SQL databases. SQL databases—such as PostgreSQL, MySQL, and SQL Server—are the industry standard for
Some time ago, I was building a chat application using AWS Websocket API gateway. Things were going smoothly. I created a WebSocket API Gateway, added $connect, $disconnect, and sendMessage/addGroup routes. From the frontend (React) side, everything was fire-and-forget. You send a message, and the onMessageHandler takes care of it 💪🏼 But then a new requirement of uploading files using S3 signed