Application Development

April 24 , 2024 by Justin Hayes

Introduction

In the past few years, the era of Generative AI (Gen AI) has emerged and its impact on the technology landscape cannot be understated. Organizations both large and small are in the process of deciding what they will do with Gen AI and how they will do it. This applies especially to product builders, as they seek to enrich their users’ experiences by infusing Gen AI capabilities into their products. It also applies to enterprise IT teams who aim to improve the productivity of their knowledge workers.

But all of these technologists face the same dilemma – whether to build their Gen AI capability in-house or buy an existing solution. This “Build vs Buy” decision is not new. We have seen it play out with open-source projects such as Linux, Kubernetes, and the big data ecosystem. We experience it regularly when deciding whether to simply use one of the cloud service offerings or manage it ourselves if we need a database, web server, messaging system, etc. 

Much like these other technology areas, the stakes can be high when making this decision. There are serious implications in terms of time to market, total cost of ownership, opportunity cost, and risk. 

This blog post is the first in a two-part series that explains how the Build vs Buy decision is playing out in the Gen AI space. Part 1 will explain each approach and share their respective pros and cons. It will also present some high-level considerations to keep in mind when making a decision. Part 2 will provide a deeper look into what each option entails, with the goal of ensuring that you are armed with enough information and perspective to make the right decision.

Gen AI Level Setting

While relatively new, Gen AI is already a somewhat broad category, spanning textual data as well as images and videos. It is used in both personal capacities and within the enterprise context. It can create art, write programs, help people find and create knowledge, ultimately even changing how humans fundamentally interact with computer systems.

This discussion focuses on the usage of Gen AI for knowledge management purposes, and for improving how people interact with computers. Exemplifying this are product builders who seek to use Gen AI to enhance product development and customer experience, and IT leadership who need to boost knowledge worker productivity. These drivers are horizontal, applying across all industries. Similarly, the dynamics behind the Build vs Buy decision are equally universal. The two approaches are explained below.

Option 1: Building Your Own Full Stack Gen AI Solution

Due to the technical complexity of the Gen AI space, a significant amount of lower-level infrastructure is required to provide quality outputs. This includes vector embedding models, vector storage, information retrieval systems, re-rankers, LLMs, GPUs, and orchestration frameworks to tie it all together. 

As such, the “Build” approach to Gen AI requires selecting, integrating, maintaining, and evolving both the underlying platform infrastructure as well as the business applications which use that infrastructure to provide capabilities to the end user. 

Pros

• Customization and Control: Building the complete stack top-to-bottom lets you tailor the entire Gen AI platform to specific organizational needs. Many components of the stack have sensible defaults that let you prototype quickly, while retaining the option for customization. 
• Integration Flexibility: Owning the complete stack provides you with the opportunity to have seamless, low-level integrations with existing systems.
• In-House Expertise Utilization: If there is already some existing talent and expertise within the organization, that can be leveraged. At this point, most companies have ML teams, so there is an opportunity to expand their charter.

Cons

• Technical Complexity: This option requires a skilled team for platform-level development, integration, and administration. While there is a small amount of skillset overlap between platform developers and business application developers, these are mostly distinct personas. Therefore it is required to find – and pay – a larger team of technical specialists, covering Data Science, ML Ops, Data Engineers, DevOps, and Security.
• Ongoing Maintenance and Upgrades: Responsibility for continuous management and improvement of the system as usage levels change and as the broader Gen AI space rapidly evolves. 
• Open Source Risk: It is common to use open source components when building your own Gen AI stack. These are typically easy to prototype but hard to productionize, to which many users of LangChain, Weaviate, and any number of Hugging Face models will attest. Also, if you leverage open source then you are at the mercy of the community to offer help and to resolve issues. This is fine… until it is not… and you are in the middle of a production outage with nobody to call.
• Higher Initial Costs and Timeline: Significant investment in development resources, tools, and lead time just to get to a state of initial operating capacity before you can offer any benefits to your end users. A side effect of building the full stack solution is that the project becomes larger and more complex, carrying a greater risk of failure and delays.

Option 2: Buying a Gen AI Platform Service on which to Run your Applications

In this model a third party provides the Gen AI platform infrastructure, and the developer is only responsible for building the applications that implement their business requirements. These applications are run on top of the platform, which is pre-integrated, tested end-to-end, and built in a way that new capabilities are exposed automatically to the application developer.

Pros

• Rapid Deployment and Functionality: Quick implementation and faster time to value due to leveraging an existing platform.
• Less Expensive Team: Skillset requirements are lower and fewer experts are needed, resulting in a cheaper team that is faster to build (and to re-build, which is inevitable thanks to the high attrition rates inherent to every technology boom cycle).
• Vendor Support and Maintenance: Ongoing technical support, bug fixes, and feature upgrades provided by the platform provider simplify ongoing management and serve as an insurance policy against technical catastrophe.
• Data and Privacy Security: Compliance with data protection laws by reputable providers who have undergone necessary security compliance requirements.

Cons

• Limited Customization: Pre-built platforms tend to be comprehensive and feature-rich, as they are made by experts in the respective space, but they may not meet all specific needs and they may not have enough flexibility.
• Dependence on Vendor: Reliance on the vendor for ongoing support and updates can be a risk in terms of lock-in and architectural optionality.

Considerations for Decision-Making

In addition to the pros and cons listed above, the following considerations need to be understood before making the Gen AI Build vs Buy decision.

• Cost: Assessing the financial implications of both building and buying. This includes software, hardware, and labor costs.
• Talent and Infrastructure: Evaluating the availability of skilled professionals and necessary infrastructure.
• Timeframe: Considering the timeline required to build the necessary team and deploy the solution, and evaluating the expected ROI.
• Compliance and Security: Understanding the implications for data privacy and security, factoring in your organizational maturity in terms of data and AI governance.
• Customization: Deciding how much customization is required to meet specific business needs.
• Scalability: Will it scale to support the current and future business usage requirements?
• Evolution: How will you keep up as business requirements change and the technical landscape evolves?

Vectara Gen AI Platform

Vectara is an example of a platform that organizations can use to simplify their Gen AI application development and operations. It provides a pre-integrated, tested, and hardened end-to-end Retrieval Augmented Generation (RAG) system on which to run Gen AI applications. There is a particular focus on mitigating the risks associated with hallucinations, by grounding all generated content on factual data specified by the user and by providing transparency into the risk posed to a given application by hallucinations.

Vectara ensures the reliability and accuracy of generated outputs by reducing the probability of inaccurate or irrelevant information. It emphasizes powerful retrieval capabilities, making it suitable for businesses that need relevant, accurate, and trustworthy Gen AI.

Finally, it provides a compelling alternative for organizations without the resources or expertise to build their own Gen AI platforms. Benefits include rapid deployment, reduced development effort, continuous vendor support, and advanced capabilities without significant investment and risks.

Case Study

One example of an organization that chose the Buy approach to Gen AI is SonoSim, a provider of ultrasound educational equipment and content. They opted to use Vectara’s Gen AI platform so they could focus their efforts on:

• their users – medical professionals and students
• their data – ultrasound educational content and simulation documentation
• their application – teaching ultrasound use to improve health outcomes

By not having to build the RAG infrastructure they got to market faster with their new features. They were able to spend their time thinking about how to structure their data for optimal result quality and what sort of user experience to offer; they were not bogged down in the Gen AI platform plumbing. Finally, they benefit from new capabilities automatically and for free – e.g. better LLMs, hallucination guardrails, new re-rankers.

You can read more about SonoSim’s experience in their case study.

As another example, a company in the Legal space started its journey by doing manual setup, integration, and tuning of Azure’s Search and AI services. They then switched to Vectara because it gave better results out of the box, without having to do low-level integration and tuning of the Azure suite to get relevant answers.

Conclusion

Gen AI is already transforming how people interact with technology, and we are only in the first mile of the marathon that is to come. Businesses stand to benefit greatly, whether they are shipping product that delights their customers or providing internal capabilities that make their knowledge workers more productive.

But businesses must decide which approach to take as they begin to incorporate Gen AI capabilities into their products and applications. Understanding the options and the corresponding cost, complexity, and risk tradeoffs will help you to make the right decision for your organization.

If you want to learn more about how Vectara can help expedite your Gen AI initiatives, check out our solution pages. If you want to test Vectara, sign up for a free account here. Within 15 minutes you will see how well Vectara’s question-answering, conversational AI, and semantic search will work for your organization.