In June 2025, Gartner predicted that over 40% of agentic AI projects would be canceled by the end of 2027¹, citing escalating costs, unclear business value, or inadequate risk controls. That prediction window is now well underway.
If you work in a regulated industry, you already know which of those three will stop your project. McKinsey's 2026 AI Trust Maturity Survey found nearly two-thirds of respondents citing security and risk concerns as the top barrier to fully scaling agentic AI², ahead of both regulatory uncertainty and technical limits. And you already know the first question your model risk function will ask about any AI agent headed for production: which model is this running on, and who chose it?
Which is why nearly every enterprise AI vendor now offers Bring Your Own Model. Microsoft, Palantir, and Salesforce all ship one. Read their documentation closely, though, and the promise is narrower than the category name implies. Microsoft tells Foundry customers that BYOM models are used at their own risk, and that managing whether data flows outside their Azure compliance and geographic boundaries is the customer's responsibility³. Palantir recommends its registered models only where you cannot use Palantir-provided models for legal and compliance reasons⁴. Salesforce's BYOLLM supports four named providers, and although inference is handled by the LLM provider, the request still routes through its own Models API and Trust Layer⁵.
Every one of those is a defensible engineering position. None of them is "no vendor lock-in, full data sovereignty." The convergence is still good for buyers, as it means BYOM has stopped being a differentiator and become a starting condition. But it also means the label tells you almost nothing about what you actually control.
Choosing your own model is a procurement outcome. And a regulated enterprise needs control — and controls have to produce evidence. The gap between those two things is where most BYOM conversations quietly fail.
What bring your own model (BYOM) actually means
Bring Your Own Model (BYOM) is a capability that lets an enterprise connect its own LLM provider accounts, endpoints, or self-hosted models to a vendor platform, so inference runs under the enterprise's chosen provider, region, credentials, and policies rather than the vendor's defaults.
Confusion comes from three adjacent terms the market uses interchangeably, which mean very different things to whoever reviews your architecture:
Approach | Who owns the inference endpoint | Who holds the credentials | What a reviewer still has to verify |
Vendor-managed model | The platform vendor | The vendor | The vendor's provider contracts, retention terms, and regional footprint — all of it inherited |
BYOM / BYOLLM | Your provider account (Bedrock, Azure OpenAI, Vertex, or an OpenAI-compatible endpoint) | You | Which region resolves, what context is transmitted, whose guardrails apply |
BYOK (bring your own key) | Usually still the vendor | You hold the encryption keys | This governs data at rest, not the inference path — the two are frequently conflated |
Self-hosted model | You, on your own hardware or in your own VPC | You | Model provenance, patch and version discipline, and the operational burden you've just absorbed |
In plain terms: BYOM changes who processes your prompts and where. BYOK changes who can decrypt your stored data. Self-hosting changes who runs the model. Most vendors mean the first when they say "bring your own model,” but only self-hosting shrinks the data path.
The four control surfaces that actually matter
To evaluate BYOM holistically, separate it into four controls. Each maps to a question an examiner, an internal validator, or a third-party risk reviewer will ask.
1. Provider and endpoint: Which company processes the prompt?
This control makes the "any model" claim real or hollow. A platform that supports a fixed list of provider integrations gives you choice within its roadmap. Salesforce is the clearest published example: its BYOLLM feature currently supports Amazon Bedrock, Azure OpenAI, OpenAI, and Vertex AI as foundation model providers⁵; that’s four names, with a separate open-connector path for anything else.
A platform that accepts an arbitrary base URL for OpenAI-compatible endpoints gives you choice within your own, including self-hosted inference servers like vLLM or Ollama, and providers the vendor has never heard of.
The reviewer's question: Which contract governs this data, and did our vendor risk team sign it?
2. Region: Where does inference physically happen?
"Data residency support" is not an answer. A compliance reader is checking whether their specific jurisdiction is on the list, and a configuration that resolves to a named region is verifiable in a way that a marketing claim is not. Look for region-scoped configuration you can inspect after the fact, not a dropdown labeled "location."
The reviewer's question: Can you show me, for this agent, which region served the request?
3. Credentials: Who owns the key, and can you rotate it?
This is an underrated consideration, but it's the most telling architectural detail in any BYOM implementation. If the credential is a separate object from the model configuration (created first, validated on its own, then referenced by ID) you have a genuine separation of duties. Security owns the credential. The data team owns which workload consumes it. And you can prove a credential works before it is ever wired to a production agent.
If credentials are typed into the same form as the model name, you have a configuration field, not a control.
The reviewer's question: Who can see this key, who can rotate it, and what breaks when they do?
4. Guardrails: Whose policy applies?
Every serious enterprise has already invested in content filtering, PII detection, and topic restriction policies. A BYOM implementation that routes through your provider account should let your existing guardrail policy apply rather than substituting the vendor's. Version-pinned, with tracing enabled, is what an auditor will ask for — because a guardrail that silently updates is a control you can't evidence retroactively.
It’s worth noting that the vendor may put this on you explicitly. Microsoft's documentation states that when you use BYOM models, you are responsible for implementing your own responsible AI mitigations, such as metaprompt, content filters, or other safety systems³.
The reviewer's question: Which policy version was in force on the date of the decision we're reviewing?
These four questions have gotten sharper recently, from both directions.
On 17 April 2026, the Federal Reserve, OCC, and FDIC issued revised interagency model risk management guidance in the form of SR 26-2, which supersedes SR 11-7 and the 2021 BSA/AML model risk statement⁶ and adopts a risk-based posture tailored to an organization's model risk profile. It isexpected to be most relevant to banking organizations with over $30 billion in total assets⁶, though it may also reach smaller institutions with significant model risk exposure. Notably, the guidance states that generative AI and agentic AI models are novel and rapidly evolving and are therefore not within its scope⁷, while adding that an organization's own risk management practices should still guide the controls for anything the document doesn't cover. The agencies have said they plan to issue a request for information addressing model risk management and, in particular, banks' use of AI including generative and agentic AI⁸ (it had not been published at the time of writing). This means the burden of interpretation, and of evidence, sits with the institution.
BYOM is not "your data never leaves"
BYOM gives you control over where inference happens, which provider handles it, whose credentials authorize it, and whose guardrails constrain it. It does not mean your data stays inside your perimeter. Prompts and retrieved context still travel to whichever endpoint is configured, and with an agent rather than a chatbot, that payload is larger than people expect. It includes the user's messages, prior turns in the conversation, and tool outputs, which can contain actual rows returned from your connected systems.
Microsoft's own Foundry Agent Service documentation tells customers they are responsible for reviewing all data being shared with BYOM models and for understanding third-party practices for retention and location of data — and that managing whether data flows outside the organization's Azure compliance and geographic boundaries is theirs to manage, not Microsoft's³. That is a hyperscaler, in its own product docs, declining to promise the thing the category is often sold on.
What BYOM controls | What BYOM does not change |
Which provider processes the request | That the request leaves your platform to reach a model |
Which region serves inference | That prompts and retrieved context are transmitted to that endpoint |
Whose credentials authorize the call | That agent tool outputs — including queried rows — can form part of the payload |
Which guardrail policy applies | Your obligation to validate the model you selected |
Which provider agreement governs the data | Where the vendor logs requests for debugging, and for how long |
The honest claim is control over the inference path, not elimination of it. Self-hosted or air-gapped endpoints narrow that path considerably, but that's a deployment choice you make, not something BYOM hands you.
Model choice is the easy half. Proof is the hard half.
So: does bringing your own model satisfy model risk management requirements?
No. Not by itself.
Model risk frameworks don't ask who selected the model. They ask for validation evidence. SR 26-2 is specific about what that means: the components of model validation are conceptual soundness, outcomes analysis, and ongoing model monitoring⁷ — assessing and documenting model design and developmental testing, comparing model outputs to real-world outcomes, and evaluating whether a model still performs as expected as products, exposures, data relevance, and market conditions change.
For a third-party model, the guidance goes further. It states that sound practice involves conducting ongoing monitoring and outcome analysis to assess whether vendor models are accurate, remain fit for purpose, and continue to be reliable⁷, and that the principles of model risk management remain applicable even where the vendor withholds the underlying code, data, or methodology. In practice, that means a process for detecting when the vendor's model changes underneath you. "We chose our own model" answers none of that. It's a sourcing fact wearing a control's uniform.
The operational data suggests this is not a theoretical gap. In VentureBeat's June 2026 VB Pulse survey of 157 enterprise respondents at companies with 100 or more employees, half reported shipping an agent or LLM feature that passed internal evaluations and then caused a customer-facing failure in the past year; one in four saw it happen more than once¹¹. The monitoring picture points the same way: once agents are live with real users, only 23% of enterprises run real-time quality checks on the answers those agents produce, while another 51% monitor system health only¹². Uptime, request traces, and gateway logs tell you an agent is running. They say nothing about whether it is right.
And note why enterprises distrust their own evaluations. Asked to name the biggest weakness, more respondents chose poor alignment with real-world outcomes than any other answer, 29%¹¹, ahead of bias, explainability, or privacy concerns. The complaint isn't that scoring is too slow or too expensive. It's that the score doesn't predict what happens in production.
We ran into exactly that problem building our own query agent. After rebuilding the agent's core, users reported clear improvements: better exploration of the data and better judgment about when to ask for clarification or say the data couldn't answer the question. The metrics barely moved. Standard academic measures like Query Exact Match and Execution Accuracy were blind to the improvement, penalizing harmless quirks like minor formatting differences or extra columns while leaving genuine gains unrecognized¹⁴. A metric that produces false negatives at that rate doesn't just understate progress. It destroys the evidentiary value of the whole exercise.
The gap is evidence you can trust.
A five-step model change-control loop for BYOM
Here is the loop we'd argue every regulated BYOM deployment needs. Call it pin-swap-prove.
1. Build the baseline first
Build an evaluation set (question and expected-output pairs) for the workloads that touch regulated data. Do this while nothing is changing. An eval set assembled in response to a model swap is a scramble; one that already exists is a control.
Tie it to governed definitions, not just to a schema. VentureBeat found 57% of enterprises traced at least one confident, wrong agent answer in the past six months to missing or inconsistent business context — wrong metrics, stale definitions, absent documents — while only 25% run a governed semantic layer in production¹². An eval set built on ungoverned context measures the wrong thing.
2. Pin the approved configuration
Record the exact model identifier, the region, the credential owner, and the guardrail policy version as the approved state. "We're on Claude" is not a pinned configuration. A specific model ID in a specific region under a specific credential is.
3. Swap in a controlled lane
Change the model for one agent or one product area, not the whole tenant. This is why per-workload model assignment matters more than the marketing suggests: it turns a model swap from a platform-wide event into a scoped test with a blast radius of one workload.
4. Re-run and compare — with a metric that survives a model swap
Run the same eval set against the new configuration and compare against the pinned baseline. This step is where most change-control processes quietly break, and the reason is specific to model swaps.
A different model writes different SQL. It will pivot a table, format a date differently, name a column something else, or add a harmless breakdown column to an aggregate — while returning exactly the information the question asked for. Compare that against a reference query on exact match and the working swap reads as a regression. Our own experience was that trivial stylistic differences and harmless extra information were among the leading reasons a prediction got marked wrong¹⁴: false negatives generated by the metric, not failures in the agent.
That is a governance problem. A metric that produces false negatives on a model change will either block a model you should have promoted, or get overridden by judgment with no record of why.
The fix is a scoring function that tests for equivalence rather than identity. Ours evolved in three stages: exact execution accuracy first; then deterministic heuristics for reasonable variation, such as ignoring column name and order, or not penalizing redundant columns; and finally a hybrid evaluator that sends only the cases failing the deterministic check to an LLM judge for a final equivalence decision¹⁴. The judge works from the artifacts a human reviewer would use: the original question, the retrieved metadata, and both the reference and predicted queries and result tables, against a codified set of judging principles, and returns a verdict with the reason for it.
That last part is what a validator reads. A pass rate alone tells you very little. Which cases failed, and on what stated grounds, is the record that survives review.
5. Validate the judge before you trust the verdict
If your comparison step uses a model to score a model, your second line will ask the obvious question: who validated the evaluator? Have an answer ready, because this is where SR 26-2's effective challenge⁷ (critical analysis by objective experts with the independence, expertise, and standing to effect change) and its conceptual soundness requirement land squarely on the eval tooling itself.
The paradigm has published support. The 2023 NeurIPS paper that formalized LLM-as-a-judge found that strong LLM judges can match both controlled and crowdsourced human preferences, achieving over 80% agreement, which is the same level of agreement observed between humans¹⁵. The same paper is candid about the failure modes, examining position, verbosity, and self-enhancement biases along with limited reasoning ability. Both halves matter to a reviewer.
So treat the evaluator as a model in its own right. Ours is held to that standard: we curated a benchmark for the evaluator itself, drawn from representative positive and negative matches, and test every change to it against that benchmark so human–LLM agreement only increases over time¹⁴. Because the judge only ever sees cases that already failed the deterministic check, the risk that actually matters is false positives (a wrong answer waved through) and that is the rate we drive down.
An evaluator with its own validation record is defensible. An evaluator nobody has tested is a second unvalidated model sitting inside your control.
6. Promote or roll back, and keep the artifact
The comparison run is the validation evidence. Attach it to the model card and the approval record, along with the evaluator version that produced it. Done consistently, the next examiner question becomes a lookup instead of a six-week evidence-assembly project.
Step | What you produce | Who owns it |
Baseline | An eval set tied to a governed data product | Data team, reviewed by model risk |
Pin | A recorded approved configuration | Model risk |
Swap | A scoped change on one workload | Data team |
Compare | A pass/fail run with per-case reasoning | Model risk |
Validate the judge | An agreement benchmark for the evaluator, versioned | Model risk / second line |
Promote | An evidence artifact on the model card | Governance / second line |
This is what Alation's evaluation capability was built for. Evaluation cases are defined against a specific data product: manually in the UI, by file upload, or with an agent auditing and updating them — and Alation simulates a fresh chat for every case, comparing the agent's output against your defined standard to identify exactly where the logic breaks down¹³. Runs are repeatable and comparable over time, and the changes made to the data product surface for human review and approval, with an audit trail.
The effect is measurable rather than asserted. Against a bare data product containing only raw table and column definitions, a query agent scored 60% on a 20-case eval set; two automated improvement iterations took it to 100%, confirmed across two consecutive evaluation runs¹³. That delta is the artifact. Pair it with our AI Governance registry, where every model, agent, and tool carries an evidence-backed model card and approvals route by regulation applicability, and the loop closes: the swap produces the proof, and the proof lands somewhere an auditor can find it.
Risk-tier your models the way you risk-tier everything else
Once model selection can be set per workload rather than only per tenant, BYOM becomes more interesting than a procurement checkbox; it becomes a risk-tiering instrument.
This mirrors how the agencies themselves reason about oversight. SR 26-2 treats model purpose and model exposure as jointly determining materiality,⁷ with higher-materiality models warranting more comprehensive and rigorous oversight, and immaterial ones needing little more than identification and monitoring for conditions that could make them material later. The guidance explicitly leaves generative and agentic AI out of scope — and tells banks to let their own governance practices determine controls for what it doesn't cover.
Workload tier | Data sensitivity | Model and region choice | Eval bar before promotion |
Regulated-data SQL generation | High — customer, financial, PHI | Validated model, region-pinned to jurisdiction, dedicated credential | Full eval set, per-case review with stated reasons, documented sign-off |
Customer-facing summarization | Medium-high — reputational exposure | Validated model, guardrails version-pinned | Full eval set plus post-launch sampling |
Internal catalog search | Low — metadata scoped to user permissions | Cost-optimized model, standard region | Spot-check eval set, monitored drift |
Exploratory research | Low — no regulated data in path | Cheapest capable model | Informal; no promotion gate |
This is the same instinct behind the broader shift in agent design. We believe that the teams to achieve durable value from agentic AI will be those who grant agents specific responsibilities within clear boundaries. Model assignment is one such rule, and it's one of the few you can set declaratively and audit later.
In practice, this looks like a tenant-wide default model with deliberate overrides at the agent and product-area level — the default keeps ungoverned workloads sane, and the overrides carry your risk tiers.
Open isn't a feature. It's a control surface.
Openness matters in regulated environments for a narrower and more practical reason than the anti-lock-in argument suggests. Your validated models, your credentials, your regions, and your guardrail policies are your control environment. A closed platform doesn't just limit your options — it substitutes its control environment for yours, and asks your second line to accept the swap.
BYOM undoes that substitution. It does not, on its own, prove anything. Model choice gets you the right to select; the evaluation loop gets you the evidence to defend the selection, and a scoring method you have validated gets you the standing to defend it again the next time it changes.
The 2026 race isn't going to the enterprise with the most autonomous agent. It's going to the one whose agents got approved for production and stayed approved — because the evidence was already assembled when someone finally asked.
Alation AIOS is the open, governed, self-improving intelligence operating system for enterprise AI. Bring your own model, govern it by use case, and keep the proof where the auditor can find it.
Sources & notes
Every external claim on this page is independently verifiable. The public sources are listed here. Competitive citations were verified 28 August 2026 and should be re-verified before publication.
Over 40% of agentic AI projects predicted to be canceled by the end of 2027, due to escalating costs, unclear business value or inadequate risk controls. — Gartner, 25 June 2025 ↗ https://www.gartner.com/en/newsroom/press-releases/2025-06-25-gartner-predicts-over-40-percent-of-agentic-ai-projects-will-be-canceled-by-end-of-2027
Nearly two-thirds of respondents cite security and risk concerns as the top barrier to fully scaling agentic AI; average responsible-AI maturity 2.3 of 4, up from 2.0, with about 30% at level three or higher on strategy, governance, and agentic AI governance; approximately 500 organizations, fielded December 2025–January 2026. — McKinsey, 2026 AI Trust Maturity Survey, 25 March 2026 ↗ https://www.mckinsey.com/capabilities/tech-and-ai/our-insights/tech-forward/state-of-ai-trust-in-2026-shifting-to-the-agentic-era
BYOM models are used at the customer's own risk and are Non-Microsoft Products under the Microsoft Product Terms; the customer is responsible for its own responsible AI mitigations, for reviewing all data shared with BYOM models and third-party retention and location practices, and for managing whether data flows outside the organization's Azure compliance and geographic boundaries. — Microsoft Learn, "Bring your own model to Foundry Agent Service," updated 13 August 2026, accessed 28 August 2026 ↗ https://learn.microsoft.com/en-us/azure/foundry/agents/how-to/ai-gateway
Palantir recommends registered models only where Palantir-provided models cannot be used for legal and compliance reasons; AIP Assist and Pipeline Builder's Generate and Explain features are unsupported; registered models do not currently support markings. — Palantir Foundry documentation, "Bring your own model to AIP," accessed 28 August 2026 ↗ https://www.palantir.com/docs/foundry/aip/bring-your-own-model
BYOLLM currently supports Amazon Bedrock, Azure OpenAI, OpenAI and Vertex AI as foundation model providers; inference is handled by the LLM provider but the request is still routed through the Models API with Trust Layer features supported. — Salesforce, Agentforce Developer Guide, "Supported Models," accessed 28 August 2026 ↗ https://developer.salesforce.com/docs/ai/agentforce/guide/supported-models.html
SR 26-2, Revised Guidance on Model Risk Management, supersedes SR 11-7 (April 2011) and SR 21-8 (April 2021); emphasises a risk-based approach tailored to a banking organization's model risk profile and the size and complexity of its operations; expected to be most relevant to banking organizations with over $30 billion in total assets. — Board of Governors of the Federal Reserve System, 17 April 2026 ↗ https://www.federalreserve.gov/supervisionreg/srletters/SR2602.htm
Generative AI and agentic AI models are not within the scope of the guidance; model validation components are conceptual soundness, outcomes analysis and ongoing model monitoring; effective challenge is critical analysis by objective experts with sufficient independence, expertise and organizational standing to effect change; model purpose and exposure together determine model materiality, with higher-materiality models warranting more rigorous oversight; sound practice for vendor and third-party products involves ongoing monitoring and outcome analysis to assess whether vendor models remain accurate, fit for purpose and reliable. — Supervisory Guidance on Model Risk Management, SR 26-2 attachment (Federal Reserve / FDIC / OCC), 17 April 2026 ↗ https://www.federalreserve.gov/supervisionreg/srletters/SR2602a1.pdf
The OCC, Federal Reserve Board and FDIC plan to issue in the near future a request for information addressing model risk management generally and, in particular, banks' use of AI including generative AI, agentic AI and AI-based models. — OCC News Release 2026-29, 17 April 2026 ↗ https://www.occ.gov/news-issuances/news-releases/2026/nr-occ-2026-29.html
The AI Omnibus entered into force on 27 July 2026; rules for high-risk AI systems in Annex III apply from 2 December 2027, and for high-risk AI embedded in physical products under Annex I from 2 August 2028. — European Commission, 27 July 2026 ↗ https://digital-strategy.ec.europa.eu/en/news/ai-omnibus-enters-force
Article 50 transparency rules apply from 2 August 2026; obligations for providers of general-purpose AI models entered into application 2 August 2025; a transitional deadline of 2 December 2026 applies to certain providers of AI systems generating synthetic content that were already placed on the market before 2 August 2026, for compliance with Article 50(2). — European Commission, AI Act Service Desk, implementation timeline ↗ https://ai-act-service-desk.ec.europa.eu/en/ai-act/timeline/timeline-implementation-eu-ai-act
Half of 157 qualified enterprise respondents at companies with 100 or more employees deployed an AI agent or LLM feature that passed internal evaluations and still caused a customer-facing failure, one in four more than once; poor alignment with real-world outcomes was the most-cited weakness in current evaluations at 29% of respondents. June 2026 VB Pulse survey, self-selected sample, findings to be read as directional. — VentureBeat, 10 July 2026 ↗ https://venturebeat.com/orchestration/enterprise-ai-is-entering-an-evaluation-gap-agents-are-gaining-autonomy-faster-than-companies-can-verify-them
23% of enterprises run real-time quality checks on live agent answers and another 51% monitor system health only; 57% traced at least one confident, wrong agent answer in the past six months to missing or inconsistent business context, and 25% run a governed semantic layer in production. VentureBeat Research, June 2026, 573 respondents across five surveys, self-selected samples. — VentureBeat, 10 July 2026 ↗ https://venturebeat.com/orchestration/wall-street-is-debating-the-ai-buildout-enterprises-just-answered-86-say-their-gpus-run-at-half-capacity-or-less
Evaluation cases defined against a data product manually, by file upload, or with agent assistance; a fresh chat simulated for every case and compared against the defined standard; changes to the data product surfaced for human review and approval with an audit trail; a 20-case eval set improved from 60% to 100% accuracy across two automated iterations, confirmed over two consecutive runs. — Alation, "How to Build SQL Agents That Actually Work: Why Evaluations Matter," 10 March 2026 ↗ https://www.alation.com/blog/ai-agent-evaluations/
Query Exact Match and Execution Accuracy failed to register a substantial improvement to Alation's query agent, penalizing minor formatting differences and extra columns while leaving genuine gains unrecognized; trivial stylistic differences and harmless extra information were among the leading causes of false-negative scores. Scoring evolved from exact execution accuracy, to deterministic heuristics for reasonable variation (ignoring column name and order, not penalizing redundant or obvious columns), to a hybrid evaluator that refers only deterministically failing cases to an LLM judge for a final equivalence check, working from the question, retrieved metadata, and reference and predicted queries and results against codified judging principles, and returning a verdict with its reason. The evaluator is itself benchmarked against curated human-graded examples, with all changes tested against that benchmark to increase human–LLM agreement over time, and with particular attention to minimizing false positives. — Alation, "LLM as a Judge: Building Smarter Metrics for AI Agents," 2 September 2025 ↗ https://www.alation.com/blog/llm-as-a-judge-ai-agent-metrics/
Strong LLM judges such as GPT-4 can match controlled and crowdsourced human preferences, achieving over 80% agreement — the same level of agreement observed between humans; the paper also examines position, verbosity and self-enhancement biases and limited reasoning ability as limitations of the approach. — Zheng, Chiang, Sheng, Zhuang, Wu, Zhuang, Lin, Li, Li, Xing, Zhang, Gonzalez & Stoica, "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena," NeurIPS 2023 Datasets and Benchmarks Track, arXiv:2306.05685, 9 June 2023 (rev. 24 December 2023) ↗ https://arxiv.org/abs/2306.05685
- AI
- Data Governance
- Data Intelligence
- Digital Transformation
- Engineering
