Create LLM: integrate your own language model with Vectara
Now you can integrate the language model of your choice into Vectara while keeping existing security approvals, fine-tuning efforts, and AI investments intact. Use Vectara’s query and chat APIs while maintaining control over your model choices.
3-minute read time
Use Your Existing Models
Many companies have already invested in specific LLMs, whether for security and compliance reasons, fine-tuning efforts, or internal workflows. Switching to a new model can require additional approvals, retraining, and adjustments, which can slow down AI adoption.
With this feature, Vectara users can integrate their preferred OpenAI-compatible LLMs, including Anthropic Claude, Azure OpenAI, and self-hosted LLMs. Instead of being limited to Vectara’s built-in models, organizations can continue using their existing LLMs while benefiting from Vectara’s retrieval-augmented generation (RAG) capabilities.
Keep Security, Investments, and Flexibility
Key benefits:
- No need for additional security or compliance reviews if a model is already approved
- Preserve previous fine-tuning and model optimization efforts
- Choose the best LLM for your use case instead of being locked into a single provider
Security and compliance reviews can be time-consuming and costly. If your organization has already approved an LLM for use, this feature eliminates the need to go through that process again. By integrating an already vetted model, companies can deploy AI solutions faster and with fewer regulatory hurdles.
Beyond security, many teams have invested time and resources into fine-tuning and optimizing an LLM for their specific use case. Switching models often means losing that work and starting from scratch. With this feature, teams can continue using a model that has already been trained to meet their needs.
Adding external LLMs also makes Vectara more flexible. Instead of being locked into a single AI provider, organizations can select models based on cost, performance, or availability. If a new LLM becomes the best fit for your application, switching is straightforward, ensuring your AI stack remains adaptable.
Example: Using GPT-4 on Azure
If your company is using GPT-4 through Azure, you can integrate it with Vectara like this:
Once configured, Vectara will confirm the setup:
Now, any queries or chat requests sent to Vectara can use your custom GPT-4 model, with your own custom prompt and model settings.
Conclusion
The Create LLM feature removes barriers to AI adoption by ensuring companies can use Vectara without security delays, lost investments, or unnecessary complexity. Whether you need to deploy an approved model, keep fine-tuned improvements, or explore new AI options, Vectara makes it possible without forcing trade-offs.
For the latest documentation about creating an LLM and how to use it, have a look here.
As always, we’d love to hear your feedback! Connect with us on our forums, on our Discord or on our community. If you’d like to see what Vectara can offer you for retrieval augmented generation on your application or website, sign up for an account!
