Securing enterprise AI: Agent permissions vs. user access
Secure enterprise AI agents with Vectara: scoped tools, ABAC retrieval filters, per-user OAuth credentials, encrypted secrets, and defense in depth.

Say you build an internal agent authorized to search and answer questions based on a corpus of an organization’s Finance and HR documents. But now the agent is using sensitive financial reports in its responses to HR reps! What's the right technique to prevent your agents from accessing information that a particular user isn't authorized to see?
This post describes the security mechanisms built into the Vectara Platform and how they apply to each place your data is stored or accessed by an agent.
Principles
Defense in depth
Protection has to follow your data through every stage: the source, retrieval, model context, sessions, files, memory, and any outbound action. So Vectara's approach is to provide defense in depth. Callers authenticate with API keys, OAuth or JWT credentials, or federated identities. Vectara’s Agents API authenticates the caller, authorizes resources and operations, protects stored data and credentials, isolates the working state, and restricts what reaches the model. At no point are secrets exposed to the model, so a user can’t get access to them, whether by accident or through deliberate prompt injection.
Least privilege
Vectara’s role-based access control (RBAC) separates account-wide roles from roles on individual corpora (collections of indexed documents). Reading a corpus, indexing into it, and administering it are separate capabilities.
Agent access comes from tool configuration. An agent can reach a corpus only through a query or an indexing tool. You can specify the corpora your tool works with. If no corpus is specified, all corpora in your account are covered. So a query tool without a specific corpus will allow queries on all available corpora, and an indexing tool without a specific corpus will allow the agent to index into any customer corpus. Least privilege means giving the agent only the tools it needs and naming the corpora for each one.
Responsible operation
If a Finance analyst leaves the company, revoking their access in your source system may leave copies in sessions, artifacts, or memory. Vectara's lifecycle mechanisms help you clean up agent context. Deleting an agent marks the agent and its sessions for deletion, and session cleanup removes their events and session files (artifacts). For widget-backed end-user sessions, the widget connector's session_tti_minutes setting expires idle sessions.
Platform security architecture
Corpora: Protecting indexed knowledge
Apply the principle of least privilege by treating each corpus as a separate tenant or dataset boundary and giving an agent tools for only the corpora it needs. Inside a corpus, attribute-based access control (ABAC) lets you decide which documents can be reached by each user’s queries. You declare access fields as filter attributes on the corpus, tag each document with them, and then apply a filter per user. Queries without a filter search the whole corpus, so set one up using this method:
- Declare your access fields, such as department or group, in the corpus's
filter_attributes. At ingestion, add those fields to each document’s metadata. - When a user signs in, your backend determines their groups and builds a filter.
- Your backend creates a session with that filter in
session.metadata.corpus_filter. - The agent's retrieval tool reads the filter from the session through
argument_override, so the model can't change it:
An HR rep and a Finance analyst can use the same agent; each session supplies its own filter, so Finance documents are excluded from the HR rep’s results before reranking and generation.
A reference is only as trustworthy as the value behind it. Control authorization metadata on the server and accept requests only when it is present. Note that a filter saved at session creation may need to be refreshed as permissions change. For example, if an HR rep moves to the Finance department, their open session keeps the old filter until your application refreshes. Apply the filter across every tool and API that reaches the corpus.
Be careful when an agent has the ability to copy data into corpora. Copied data doesn’t automatically inherit the source's permissions, so either keep it in a corpus that other agents’ tools don’t name and ensure none of those tools are left unscoped, or tag it with access fields your filters enforce.
External sources: authorization follows the credential
Agents can also fetch data from external systems. Vectara provides built-in tools for Google Drive, Confluence Cloud, and many more external data sources. You can also configure a built-in web_get tool for HTTP APIs, or integrate with services via MCP. When an external system needs to authorize access, Vectara provides different credentials depending on the tool. This means it’s the identity represented by the tool’s credentials that gets authorized, not necessarily the person chatting.
- To apply the user’s permissions, configure the tool to apply the user’s own credentials. Your application runs the OAuth consent flow and stores credentials in
session.secrets, and you configure the tool to reference them. - To apply a shared service account’s permissions, accessed by all users, store and access the credentials in
agent.secrets. If users have different access rights, enforce resource and operation restrictions in trusted application/tool code or configuration.
For example, say the agent looks up tickets in Jira. If everyone uses the same Jira service account, store that token in agent.secrets and point the tool to {"$ref": "agent.secrets.jira_token"}. Jira will then use the service account’s permissions, regardless of whether the request comes from the HR rep or the Finance analyst. If each person connects their own Jira account, store each user’s OAuth token in session.secrets when the session is created, and have the tool reference {"$ref": "session.secrets.jira_token"}. Jira will then use that person’s permissions. The important part is that your application has to put the right token in the right session. Storing a secret does not do that automatically.
Encryption and data minimization
Vectara supports TLS, encryption on disk, corpus-specific symmetric encryption keys, and customer-managed AWS KMS keys. Metadata fields declared as filter attributes, including your access fields, are excluded from the separate text-and-metadata content-encryption layer.
Agent security
Agent permissions vs. user permissions
Each agent has its own service account identity that controls which Vectara resources it can access. So when an HR rep asks a question, three separate checks apply:
- Can they use this agent and session? Vectara checks their role and, on the end-user API, that they own the session.
- Can the agent perform this operation? Vectara checks the agent's service-account permissions.
- Which documents can they receive? Your application enforces this with retrieval filters or per-user external credentials.
The agent executes with its own authority. How you configure the agent will determine how a user’s data restrictions apply to the retrieval and tool paths you configure.
Secrets: Credentials outside model-visible configuration
Store credentials in secrets, not in instructions, metadata, or literal argument_override values. agent.secrets are shared across an agent's sessions; session.secrets belong to one conversation. Both are encrypted at rest, masked in API reads and session events, and resolved when the tool runs.
Note that masking has limits:
- A custom tool receives the real credential and can log it.
- A client tool's pending arguments contain resolved secrets unmasked because the client runs the tool. In a widget, the client is the visitor's browser, so never pass a server-only secret to a client tool.
- A hidden token doesn't stop an overly permissive tool from sending data elsewhere. Pin each tool's destination and method in its configuration.
Design for the conversation
The data an agent can access may differ from the data that belongs in a user's conversation. To build secure agents that you can trust, start with our Security guide.