Retrieval Augmented Generation

August 29 , 2024 by Miluska Berta

Before we tackle the question at hand, let me provide some background on my journey to Vectara and how I reached this conclusion.

My Journey

It all started two years ago while having lunch with Amr Awadallah, CEO and Co-Founder of Vectara, along with some fellow Cloudera alumni. Amr shared with us his latest project, Vectara, and of course, given all our shared experiences working together, we immediately offered to jump ship at our current organizations as soon as he had a headcount. His sheer enthusiasm about how his project is going to revolutionize the AI space was contagious and we all knew we needed to be a part of it, especially me. At this stage in my career with the dynamic experience I have as a technical marketer, my goal was to join an ideal organization with a prime leadership team. There was no doubt in my mind that Amr’s company was the one. Two years later, ZIR AI is now Vectara and they were able to generate headcount through a very successful Series A funding! I interviewed in June and officially started on July 1st.

You see how I patiently waited for this opportunity? Very demure, very mindful. 💅

From the day I started, I quickly came to realize that AI is a very broad topic. Under that AI umbrella, we are specifically in the GenAI space as an end-to-end platform with Retrieval Augmented Generation [RAG] as a Service. You know that theory about knowledge; the more you learn, the more you realize you have so much more to learn. I am humbled every day, and the more I learn about the AI ecosystem, the more I realize I don’t know much! The industry conferences I was evaluating were of no help either, since it became abundantly clear that every media company was rebranding their conferences to the latest buzzword; AI. It took me talking to practitioners at these shows and subject matter experts at Vectara to truly understand the value that could be gained by harnessing this technology.

What’s RAG All About?

Let’s first break this all down: What is RAG? For starters, we need to understand Generative AI, or as the cool kids call it, GenAI. GenAI is an AI-powered tool where you can submit a request or a question and the deep learning tool gives you an answer (think of ChatGPT, MidJourney, MuseNet, etc.) GenAI is designed to provide human-like responses in a matter of seconds. Although these tools are insanely impressive, there have been cases where the responses/answers are wrong, otherwise known in the industry as hallucinations, costing people their jobs and costing businesses unnecessary liabilities.

Hallucinations are preventable if you have the right RAG-as-a-Service in place. So what in the world is RAG?

Retrieval Augmented generation is a great supplement to GenAI tools because of its two-step approach as follows:

1. Retrieval: When RAG is under the hood of your GenAI tool and you ask it a question, it will search through any relevant data sources regardless of what type of data it is; structured vs unstructured (PDFs, video files, photos, etc.) and wherever it may live, as long as you’re providing that data source. This type of retrieval method guarantees the most up-to-date information is provided.
2. Generation: Based on this type of accurate, retrieved information, RAG will generate an answer to your question that has safeguards for hallucinations. In other words, it’ll give you the right answer, for example, those lawyers who lost their jobs when they had ChatGPT write a brief for them that was completely inaccurate would still have their jobs. Vectara even gives you a factualness score for each answer.

What I am Hearing from AI Conference Attendees…

I have two types of conversations with AI conference attendees. One is with data and AI analytics people who have expressed pain points around their GenAI tools not being able to retrieve data that’s confidential and/or unstructured. I’ll inform them that our RAG is HIPAA and SOC 2 compliant with the ability to process structured and unstructured data. The second type of conversation is with an engineer on the machine learning team. 8 out of 10 times they immediately respond to my sales pitch with “We built our own RAG”. My response is always, “I love a good DIY project just as much as anyone else, but why would you exhaust all your technical resources to build and run a DIY RAG service when Vectara can do it all for you?”

Conclusion

Here’s my take…you shouldn’t care about RAG. You should care about results.

Understand it, yes, but think of RAG as a GPS tool and your car is a GenAI tool. You can drive anywhere, but the right GPS tool will get you there faster and more efficiently using real-time, up-to-the-minute data. All you should care about is having a RAG solution in place or in this analogy, a trusted GPS tool to get you where you need to be in an accurate and efficient way.

Let’s work smarter, not harder.