Application Development

June 06 , 2024 by Shane Connelly

Hello Vectara community!

Today, we’re excited to announce that we’re introducing a brand new REST API for Vectara: API v2.  API v2 brings enormous quality-of-life improvements for software developers using the Vectara platform and we’d like to walk you through what some of those are, why we’ve chosen to make them, and what you can do to get the most out of the Vectara platform now.

Simpler Getting Started Experience

Vectara’s API has been rich and highly capable: you can define a reranking strategy, a hybrid/keyword strategy, specify multiple corpora to target your query against, provide structured filters, and so on.  However, a complex JSON structure encompassing the various options has been required, meaning the richness of the API has made it less ideal for getting started with Vectara.  With API v2, you can now get started with a number of simple API calls: sometimes with no JSON body at all.  For example, you can now query with a simple query string:

curl -L -X GET 'https://api.vectara.io/v2/corpora/my_corpus/query?q=what%20is%20the%20meaning%20of%20life' \
-H 'Accept: application/json' \
-H 'x-api-key: <API_KEY_VALUE>'

And if you want to list the documents in a corpus, you can now do that with a simple GET request:

curl -L -X GET 'https://api.vectara.io/v2/corpora/my_corpus/documents' \
-H 'Accept: application/json' \
-H 'x-api-key: <API_KEY_VALUE>'

For those that have used API v1, you might have noticed in the above examples that there’s a new parameter in the URL: “my_corpus.”  This new parameter is a corpus key, which allows you to associate corpora by an ID of your choosing (like “my_corpus”) instead of just relying on Vectara to auto-increment a numeric ID, as has been the case in API v1. Old corpora have been automatically assigned a corpus key derived from their name and corpus ID.  You can find a list of your existing corpora and corpus keys in the corpora list once you’ve logged in.

In addition to the API structure, the authentication flow is also much simpler now.  In v1, every individual account had an OAuth authentication URL  of https://vectara-prod-[your_customer_id].auth.us-west-2.amazoncognito.com/oauth2/token, which meant you’d have to modify it from any example code.  In v2, there’s no need for any “special” OAuth URL for each user: it’s standardized to https://auth.vectara.io/oauth2/token.

More REST-ful

The next thing you’ll notice is that Vectara’s v2 APIs are much more RESTful, and follow the resource-oriented design methodology.  They use the standard HTTP verbs POST, GET, PATCH, and DELETE for CRUD operations, and the URL structure is much more object-oriented to the objects that exist in Vectara.  For example, GET https://api.vectara.io/v2/users/:username will retrieve details on a user while DELETE https://api.vectara.io/v2/users/:username will delete the same user.  Users, corpora, queries, documents, chats, asynchronous jobs, LLMs, rerankers, API keys, and more are now top-level object structures that you can use in this fashion with these standard HTTP verbs.  This should make it much more intuitive to get started developing your application on top of Vectara.

Standard Error Codes

Continuing with being more RESTful, the REST APIs in v2 also now make use of standard HTTP error codes.  For example, if you get a 200 OK, things are…OK!  A standard 404 is returned if you try to delete a corpus that doesn’t exist and a 403 is returned if there’s a permissions error in attempting to update or delete something the user doesn’t have access to.  While Vectara did return error details in v1, it was typically placed inside a JSON payload in the return message, and not all users were successfully looking at it and parsing it out.  This led to a number of “silent” errors for users as they were building their apps.

Better Out-of-the-Box Defaults

Over the past couple of years, we’ve just learned more about optimal values for things like how hybrid search works best in practice with our embedding model and exactly when and how to do reranking.  The APIs in v1 have supported a long list of configurable options, but in some cases, we’ve learned about better default values for those options.  For example, when we launched hybrid search, we first found that the sweet spot for lambda was typically about 0.025, but we’ve since learned that Boomerang typically has a slightly lower optimal value for most use cases: something closer to 0.005 to 0.01.

In both cases, we couldn’t update the default values without making a breaking change in the API: moving to a non-zero value would have been a breaking change for those users that only wanted neural retrieval, and even if we had set a non-zero value, changing it to a different non-zero value would also have been a breaking change.

In v2, we’ve done two things to help address this.  First is that we’ve updated the default values with everything we’ve learned from operating a semantic search and GenAI platform at scale.  Second, we’ve intentionally under-defined certain parts of the API in v2.  For example, in v2 you can define a particular lambda value, etc, but if you don’t, we’ll continuously make improvements to the platform to provide better, more relevant results based on everything we continue to learn going forward.

The Path Forward

We’ve heard the feedback that developers had on Vectara’s APIs and hope this goes a long way to help address some of the concerns.  We also want to use this opportunity to clarify a few additional points for the APIs.

First, many of you use gRPC, which Vectara supports in addition to REST.  We want to let you know here that we have no plans to remove gRPC: v2 of the REST APIs is all about improving the experience for the large number of users we have using REST as opposed to being the “only” path forward.  gRPC is still a great way to develop low-latency applications on Vectara.

Second, we wanted to mention that all of this is fully baked into a public OpenAPI specification.  You can download that spec and use a tool like Postman, Insomnia, or another tool of your choice to build and test the APIs with Vectara.  We’ve integrated the OpenAPI specification directly into our API playground so that it stays up-to-date as we roll out new features.  Simply select “2.0” from the API Playground version drop-down on the docs to ensure you’re looking at v2.

Finally, this is by no means the end of our quest to make Vectara the easiest GenAI platform to build on top of.  In the future, we plan to take this new API structure and release native language clients for popular programming languages used in the enterprise.

Conclusion

We’re very focused on the developer experience here at Vectara and have heard the community feedback on our REST APIs.  We’re really proud of the work we’ve done to get a new, better API out the door!  We’re excited to see what you’re able to build.

As always, we’d love to hear your feedback! Connect with us on our forums or on our Discord.

If you don’t have an account already, sign up for a free Vectara account to see how Vectara can help you easily leverage RAG in your GenAI apps!