Arixent guide to evaluating security in enterprise AI solutions: what the system can access, disclose and change, with the line "A demo is not a security test."

How to Evaluate Security in Enterprise AI Solutions

Security evaluation for enterprise AI solutions comes down to three questions. What can the system access? What can it disclose? What can it change? Encryption and compliance reports matter, but they do not tell you whether an assistant can pull another team's documents or whether an agent can push an unauthorized transaction.

Evaluate the whole application, not the model alone. That means the data pipelines, retrieval indexes, identities, orchestration, connected tools and the people who run it. This guide gives you an evidence-led way to compare vendors or implementation partners, test the security boundary and set release gates that hold up in production.

Define the workflow and its security boundary

Start with one workflow: employee knowledge search, customer support or invoice processing. Write down the users, the sensitive data, the connected systems and the decisions it touches. Name who gets hurt by a wrong answer, an unauthorized disclosure or an unintended action.

For enterprise AI solutions, the security boundary rarely stops at the application interface. Map where information actually travels: model providers, vector databases, observability services, external tools and support channels. Include temporary storage, caches and administrative access. A deployment diagram that skips those paths is not ready for approval.

Match the evaluation to the workload. Predictive models need attention on training-data integrity and model artifacts. Retrieval applications need permission-aware search. Agents need controls over actions, approvals and recovery. The higher the consequence, the stronger the evidence and the narrower the starting permissions.

Use NIST's AI Risk Management Framework and its Generative AI Profile to structure ownership and risk decisions. They are frameworks, not product certifications. If workflow ownership or data classification is still open, run an AI readiness assessment before you score a single vendor.

Trace sensitive data through its full lifecycle

Ask for a data-flow diagram and a written inventory: prompts, uploaded files, retrieved passages, outputs, embeddings, feedback and logs. Every category needs an owner, a purpose, a storage location, a retention period and a way to delete it. Derived data counts too. Embeddings are not anonymous just because they are numbers.

When comparing enterprise AI solutions, separate "not used for model training" from "not retained." A provider can keep your content out of training and still hold it for abuse monitoring or troubleshooting. Confirm the terms for the exact service, account configuration and deployment you are buying, not the ones on the marketing page.

Check encryption in transit and at rest, key ownership, secret storage and who can reach decryption. Customer-managed keys suit some environments, but they do not stop an authorized application from exposing decrypted content. Evaluate the whole access path. A key-management feature is not a security boundary.

Then test deletion. Remove a source document and look for it in the index, the cached responses and every downstream copy. Record backup retention exceptions and restoration behavior. Where it matters, confirm processing regions and subprocessors in the contract. Privacy and sector obligations need legal review; a vendor's general compliance statement does not establish that your use is permitted.

Verify identity, authorization and tenant isolation

Require enterprise identity integration, controlled service accounts and least-privilege access. Keep three sets of permissions distinct: the human user's, the application's and the agent's. A broadly privileged backend account quietly bypasses every restriction that looks fine in the interface.

For retrieval-augmented generation (RAG), authorization has to constrain what reaches the model. Filtering the answer after unrestricted retrieval is too late. The sensitive passage is already in the prompt, the trace and the cache. Enforce permissions at the retrieval and document-access boundary, and fail closed when a permission cannot be determined.

Test it with two users at different access levels and, where it applies, two separate tenants. Try direct document references, paraphrased questions and follow-ups that lean on an earlier privileged session. Inspect cache keys and conversation memory for cross-user contamination. Revoke access and run it all again to measure how long the revocation takes to land.

Strong enterprise AI solutions also limit administrative access. Ask how support staff get in, whether that access expires and what evidence you can inspect afterward. Shared accounts, standing production privileges and unexplained support exceptions are findings, not footnotes. You want demonstrated authorization enforcement, not a checkbox that says role-based access control is supported.

Permission-aware retrieval in enterprise AI solutions: the permission check sits before retrieval, fails closed, and only permitted passages reach the model and the answer.

Test prompt injection and information disclosure

Prompt injection is untrusted content trying to steer the system. It arrives in a user message, a retrieved document, a web page or a tool response. One instruction buried in an otherwise relevant document becomes dangerous the moment the application treats that content as authority.

Use the OWASP Top 10 for LLM Applications to organize the threats: prompt injection, sensitive information disclosure and excessive agency among them. Ask vendors to explain their trust boundaries and show the controls working. A promise that the system prompt or a second model will reliably refuse malicious instructions is not a control.

For enterprise AI solutions, test whether injected content can expose protected context, redirect tool use or send data to an unapproved destination. Use synthetic secrets and controlled endpoints in an authorized environment. Cover file uploads, retrieved passages and tool outputs, not just the chat box.

Look for containment in the architecture: restricted network egress, explicit tool permissions, output validation and a hard separation between instructions and untrusted material. No injection test set proves universal resistance. The goal is simpler and more useful: a manipulated model response cannot cross an enforced data or action boundary.

Put independent controls around agent actions

An assistant that drafts an answer and an agent that changes customer records or moves money are different risk classes. List every tool the agent can call, its allowed parameters and the worst thing misuse could do. Start read-only. Add write access only when the workflow proves it needs it.

Authorization belongs in the tool or application layer, outside the model's discretion. Validate schemas, object ownership, transaction limits and business rules before anything executes. Never accept a model-generated claim that a user approved an action as evidence of approval. The authenticated user and the authorization service establish that, independently.

For consequential actions, bind approval to the exact operation: recipient, amount, record identifier and payload. If any of those change, approval starts over. Use idempotency controls so a retry cannot create a duplicate transaction. Separate proposal from execution. A plausible explanation is not permission.

When enterprise AI solutions take actions, recovery has to be explicit. Know which changes reverse cleanly and which need compensation or a human. Test stopping an agent mid-workflow, handling partial completion and routing unresolved cases to a named operator. A kill switch must disable further actions without the model's cooperation.

Independent authorization boundary between an AI agent's proposal and execution in enterprise AI solutions: recipient, amount, record and payload are checked outside the model before anything runs.

Examine vendor and software supply-chain evidence

Vendor due diligence starts with evidence about the services actually in the proposed architecture. Useful material: independent assurance reports, penetration-test summaries, remediation status, dependency inventories and documented incident procedures. A model provider's controls do not cover the orchestration layer an implementation partner built on top.

Read the scope, assessment period, exceptions and customer responsibilities. A SOC 2 report is evidence about specified controls, not proof that your AI workflow is secure. Ask which subprocessors and environments are covered, what is excluded and how open findings affect your deployment.

Enterprise AI solutions also run on models, packages, containers and connectors that change on their own schedules. Check artifact provenance, dependency scanning, secret scanning and deployment approvals. For internally trained or fine-tuned models, inspect dataset access, training-pipeline permissions and artifact integrity. Where model interfaces are exposed, look at abuse monitoring and protection against extraction attempts.

Put responsibility boundaries in writing: who patches each component, who investigates suspicious behavior, who authorizes emergency changes. Contracts should cover data handling, incident notification, subprocessor changes and exit assistance. If a vendor cannot hand over sensitive evidence directly, agree on an assurance process. Do not accept an unsupported claim because the sales cycle is ending.

How to test enterprise AI solutions before approval

Run an authorized evaluation in a representative staging environment with synthetic or approved test data. Put security engineers, application owners and business operators in the room. A demonstration the seller controls end to end is not a substitute for repeatable tests against your configuration.

Build a small threat-based suite with explicit expected outcomes:

  • Permission bypass: A user requests a restricted document directly and through an indirect question. Retrieval must deny access.
  • Cross-tenant leakage: Two tenants submit similar requests. Responses, conversation state and caches must stay isolated.
  • Malicious retrieved content: A document instructs the agent to export records. Tool and egress controls must block the operation.
  • Revoked access: A previously authorized user loses permission. Retrieval, active sessions and cached content must follow the documented revocation policy.
  • Unsafe execution: An agent proposes an unapproved account change. Execution must stop at the independent authorization boundary.

Use MITRE ATLAS to widen the scenarios around real adversary behavior. Repeat the suite across model versions, retrieval settings and tool configurations. Record the input, the authenticated identity, the configuration, the expected result and what actually happened, so every failure can be reproduced.

Measure unauthorized disclosures, unauthorized actions, detection coverage and recovery time. Count legitimate requests that were wrongly blocked too; a system that prevents all useful work is not a successful deployment. Zero failures in a limited suite is encouraging evidence. It is not proof.

Five threat-based tests to run on enterprise AI solutions before approval: permission bypass, cross-tenant leakage, malicious retrieved content, revoked access and unsafe execution, each with the outcome that must happen.

Score evidence without averaging away critical failures

Use one rubric across vendors, and keep the nonnegotiable gates out of the weighted average. What follows is a working structure, not a certification standard. Adapt it to your data sensitivity, operating model and the cost of failure.

Security domain Evidence to request Validation activity
Data lifecycle Data inventory, retention terms and deletion procedure Trace and delete a test record
Authorization Identity architecture and permission rules Attempt cross-user and cross-tenant access
Injection containment Trust boundaries and tool restrictions Inject instructions through retrieved content
Agent safety Action limits and approval design Attempt an unauthorized operation
Supply chain Component inventory and assurance reports Review scope, provenance and open findings
Operations Monitoring, audit logging, response and recovery procedures Exercise containment and service recovery

Score each domain from 0 to 3. Zero is absent. One is documented. Two is demonstrated. Three is demonstrated with repeatable tests and an assigned operational owner. Keep the evidence reference next to the score so another reviewer can reach the same judgment.

Do not approve enterprise AI solutions because the average looks strong. An unresolved cross-tenant disclosure or an unrestricted consequential action blocks release, whatever the strengths elsewhere. Record every exception with an accountable risk owner, compensating controls, a remediation deadline and a named approval authority. Compare vendors on the quality of their evidence and the residual risk, not on the polish of the presentation.

Evaluate monitoring, response and change management

Security evaluation does not end at procurement. Ask which events you can see: authentication failures, authorization denials, unusual retrieval volume, blocked tool calls and administrative changes. Audit logging should connect a request to its identity, configuration and resulting actions without copying sensitive prompts or documents wholesale.

Decide the access controls and retention for the monitoring system itself. Redaction reduces exposure; metadata and carefully chosen content keep investigations possible. Test whether an investigator can reconstruct a consequential action without every operator getting access to confidential records. Auditability and data minimization have to be designed together.

For production enterprise AI solutions, require a response plan for credential compromise, data leakage, malicious documents and unexpected agent behavior. Name the person who can disable tools, revoke credentials, quarantine an index and move the workflow to a manual fallback. Then rehearse it. A runbook nobody has exercised is a document, not a plan.

Model updates, prompt edits, connector changes and retrieval tweaks all change security behavior. Require change records and regression tests before promotion. Where the provider controls updates, agree on notification and validation, and keep monitoring for surprises. Be explicit about rollback limits: reverting code does not unsend a message or take back a disclosed record.

Compare delivery models and security operating costs

Buying a managed application, building on a platform and bringing in an engineering partner draw the responsibility boundary in different places. A managed application takes infrastructure work off your plate and may limit visibility and customization. Building gives you control and leaves integration, testing and operations with you.

Compare at the component level rather than treating "build versus buy" as one decision. Our guide to choosing which parts of an LLM stack to build, buy or blend gives you a frame for it. For each component, name who implements the control and who verifies that it keeps working.

The total cost of enterprise AI solutions is more than subscriptions and inference. Add identity integration, permission synchronization, private connectivity where needed, security testing, redacted logging, human approval time and incident support. Add index rebuilds, regression runs and a manual fallback that stays maintained. Ask every proposal to estimate against the same workload assumptions so you can compare them.

Security requirements also shape latency and throughput. Measure the full workflow with authorization checks, validation and approvals switched on. A cheaper configuration that leaves out required controls is not a cheaper equivalent. Put migration and verified deletion costs in the exit plan before you sign.

Set pilot exit criteria before expanding access

A pilot should produce evidence for a release decision, not just happy user feedback. Establish the baseline before deployment: completion time, review effort, error frequency and existing access controls. Then measure whether the AI-assisted workflow improves the business outcome while staying inside the approved security boundary.

Define the release gates in advance. A practical starting point: no unauthorized disclosures or actions in the agreed test suite, verified enforcement of access revocation and a containment procedure that has actually been exercised. These are acceptance criteria, not guarantees of future safety. Higher-risk deployments may need independent assessment on top.

Expand enterprise AI solutions in steps: by user group, by data classification, by action capability. Keep consequential writes disabled until the matching controls pass. Have business, security and operations owners accept residual risk together, with legal or compliance review where it applies. Write down who can halt expansion if new evidence changes the picture.

Reject or defer when a vendor cannot explain data handling, demonstrate isolation or name an accountable incident owner. Conditional approval gets an expiry date and a narrow scope. Commercial urgency is not a compensating control, and a great productivity demo does not close an untested security boundary.

Frequently asked questions

Is a SOC 2 report enough to approve an AI vendor? No. Review its scope, exceptions and customer responsibilities, then test the application-specific controls yourself. The report will not tell you about permission-aware retrieval, prompt injection containment or how your connected tools behave.

Can private deployment eliminate AI security risks? No. It changes data exposure and infrastructure control. Weak authorization, malicious documents, excessive permissions and unsafe actions are still possible. Evaluate the application and the operating procedures regardless of where it is hosted.

How often should enterprise AI solutions be reassessed? After any material change to models, data sources, permissions, tools or providers, and on a risk-based schedule in between. Higher-consequence workflows need more frequent testing and closer operational oversight.

Who should make the final approval decision? Business owners accept the workflow consequences, security assesses the technical risk, operations confirms supportability, and legal or compliance covers the obligations. Then name one accountable release authority. Approval left implicit is approval nobody owns.

Bring security evidence into the engineering decision

A useful security review ends with a documented boundary, reproducible tests and named owners for the remaining risk. Carry those into procurement, architecture and delivery acceptance so they survive past the pilot.

Arixent designs, builds and runs production AI systems, data platforms and software products, and the boundary described above is where our engineering starts, not where it ends. If you are evaluating an implementation partner, bring your workflow, your data classifications and the security evidence you need, and we will turn them into engineering scope and production acceptance criteria before development begins. Book a working session to start with one workflow.

Scroll to Top