The GenAI routing layer is the least-documented part of the pharma stack

When an AI assistant in our company sends a question to a model, who decides which model it goes to, who can see what was sent, and what should I ask for to find out?

The routing layer is the piece between the chatbot and the models. It picks the model, decides what happens when one is down, and blocks models nobody approved.

Who can see what was sent is a question for the audit log. It is one of the five questions at the end.

flowchart LR
  App["Application<br/>chatbot or agent"] --> Layer["Routing layer<br/>model, fallback, block"]
  Layer --> Models["Approved models"]

What this is, and what it is not. This is a gap in what companies say in public. It is not a finding about what they run. The Pharma AI Tracker lists an initiative only when a public source describes it, and it keeps a verbatim quote. The method page states the rule the list itself uses: if it isn’t disclosed, it isn’t here. A missing row means there is no public source the Tracker can quote. It does not mean the company has no gateway.

Who this is for and how to use it

This block is my interpretation of how to use the note. It is not a claim about any company.

• If you lead the AI platform, or you decide which AI projects go ahead, use it to decide what to ask before you approve the next one.
• If you own computer system validation (CSV) or IT quality, use the five questions as a first-pass review list for anything that calls a model.
• If you lead quality, ask to see answers to those five questions before an AI project moves ahead.
• If you build or sell the product, expect to be asked these five questions.

Plain-English glossary

• Routing layer. Also called an LLM gateway or an AI gateway. The piece between an assistant and the models. It picks the model, handles a model that is down, and blocks models nobody approved.
• Control layer. A shared set of controls in front of more than one model or agent, rather than a rule that lives inside one chatbot.
• Fallback. What the routing layer does when the first model is down. It may send the call to another model, or it may stop the call.
• Intended use. The job a system is meant to do. In this note, validation follows that job.
• GxP. The “good practice” rules used in pharma. Work done under those rules is GxP work.
• CSV and CSA. Computer system validation and computer software assurance. Both are ways of showing that a computer system fits its intended use. CSA is the subject of FDA’s Computer Software Assurance guidance.
• GAMP Category 5. The GAMP category for software built custom for one company. A custom routing layer is likely to sit here.
• ALCOA+. A data-integrity checklist for records: attributable, legible, contemporaneous, original, and accurate, plus complete, consistent, enduring, and available.
• Audit trail. A record of who did what, and when, that also shows if someone later changes it.

What the public record shows

The Pharma AI Tracker has 349 entries across 49 companies, the 49 largest pharma companies by market cap. The default view hides research partnerships. It shows 261 entries across 45 companies, and it holds 88 entries back. Four companies appear only on those hidden rows.

A search of all 349 entries, the initiative name, the summary, and the stored quotes, for gateway, routing, router, model-agnostic, Unity AI Gateway, LiteLLM, or Portkey returns nothing. That zero counts those words in the Tracker text. It is a count of public disclosure, not a count of systems in use.

The 23-company split below is my own reading of the entry text. It is not a field in the Pharma AI Tracker.

On that reading, 23 of 49 companies have at least one entry for an internal GenAI platform or a company-wide assistant. All 23 sit in the default view. Those entries add up to 37. Thirty-four are the assistant or the platform. Three describe access to more than one model, or a control layer. Twenty-six of the 49 companies have no such entry. In the default view, that is 22 of the 45.

The three that come closest

If you are looking for a shared door in front of several models, three entries are as close as the Tracker gets. All three are in production. Takeda and Merck & Co. are labeled unclear. Novo Nordisk is labeled non-GxP. Only Takeda’s Tracker summary uses the words “control tower.” That phrase is not in the source quote.

Company	Public record	Stage	GxP
Takeda, Self-Service AI Foundation	Agents on Databricks for 10,000-plus employees. The Tracker summary adds a control tower, a policy-enforcement agent, and an agent marketplace.	Production	Unclear
Merck & Co., GPTeal	An in-house workspace. The summary says it offers several large language models. The quote does not name a router.	Production	Unclear
Novo Nordisk, Bedrock chatbot platform	Employees build chatbots on Amazon Bedrock models. This is a vendor case study, so treat it as multi-model access on a cloud service, not as a named in-house control layer.	Production	Non-GxP

Takeda. A Databricks conference page places the foundation on the “Databricks Data and AI platform” and says it “enables 10,000-plus employees to develop, deploy, and operate AI agents”. Takeda also has myAibou, in production, labeled non-GxP. The company page limits what that assistant can see to “what the user gives it in their question or prompt”.

Merck & Co. A Fortune interview says “43,000 employees are using GPTeal today” and that they have “entered 80 million prompts”. The line about several models is the Tracker’s summary, not the quote.

Novo Nordisk. The source is an AWS case study. Employees “use the foundation models in Amazon Bedrock to build and customize chatbots”. Treat this as multi-model access on a cloud service in a vendor source, not as a named in-house control layer.

What most companies disclose instead

The other 34 entries name an assistant or an internal platform. Some are chat tools, some are places to build AI, and some are counts of agents people made. Eight of them are below. Where a line says summary, that is the Tracker’s summary, not the source quote.

Entry	What the source says	What it leaves open
Sanofi, AI Foundry (Sanofi)	“enables data and AI builders to innovate responsibly at scale”. Summary: an internal platform for building AI.	Whether any of that building includes routing. The quote is silent on it.
Merck KGaA, myGPT Suite (2025 annual report)	“more than 90% cost savings compared with off-the-shelf options”. Summary: an internal platform with more than 32,000 active users a month.	The public detail is cost and headcount.
Pfizer, Vox (AWS)	A “Pfizer-certified generative AI platform”. Vendor source. This is the Pfizer platform entry.	The source names the platform and stops there.
Moderna, mChat (Moderna)	“launched mChat in May 2023”. Summary: an in-house chat tool.	A launch date for a chat tool.
Daiichi Sankyo, DS-GAI (Daiichi Sankyo)	“chat responses based on files and internal data, and image generation”. Summary: an in-house generative AI system.	What the chat can do.
Sanofi, Concierge (SEC filing)	“access to over 20,000 data points, supports over 30,000 users”. Summary: an internally hosted companion.	Those data points are records, not models.
AstraZeneca, AI Agent Builder (AstraZeneca)	“over 1,500 agents have been developed in the first two months”, inside the Thriving in the Age of AI programme.	This is an enablement count. It is a count of agents people built, not a model router.
Vertex, VAIDA (Microsoft)	“orchestrates multiple AI agents”. Vendor source.	VAIDA lines up its own agents as one assistant. A shared layer would sit in front of other applications.

Vendor products sit in the same 37. Amgen, ChatGPT Enterprise (Amgen), in production: “access was scaled across the company”. Amgen, Microsoft 365 Copilot (Amgen), in production: “grown to 20,000 employees with access”. Bristol Myers Squibb and Anthropic (Bristol Myers Squibb) is still announced. The source says “advanced reasoning and agentic capabilities”, and the Tracker summary says the agreement covers more than 30,000 employees.

The Tracker shows home-built assistants and vendor products side by side. Read that as a list of what was disclosed. It is not a build-versus-buy finding.

Big announcements can still miss this layer

The 22 companies in the default view with no platform or assistant entry in this reading include Eli Lilly, Roche, Novartis, AbbVie, and CSL. Several of them have other entries. Those entries are about something else.

Eli Lilly, LillyPod is a GPU system. The Tracker summary describes an NVIDIA-built supercomputer for scientific research. It sits on a research-partnership row, so it is one of the 88 hidden by default. Roche, NVIDIA AI factory is the same kind of row. The Roche release says the footprint “now exceeds 3,500 Blackwell GPUs”. CSL and AWS (CSL release) is announced and is in the default view. The summary describes AI and cloud technology for research and clinical development, and names Amazon Bedrock among the services. Novartis and AbbVie have other entries too. None of these is a GenAI routing layer.

For those 26 of 49, and the 22 of 45, no such entry means the Tracker has no qualifying public source. The company may still have a gateway, a platform, or a validated layer. The Tracker cannot see what was never disclosed.

Where the detail goes instead

Where the Tracker is specific, it is specific about an application. Johnson & Johnson, clinical trial report preparation (Reuters), in production, GxP label unclear, puts the time change at “700 hours to about 15 minutes”. Takeda, AI investigation digital assistant for deviations (BioProcess Online), in production, labeled GxP, is there to “enhance report standardization and improve the quality of investigations”. The application carries a figure or a job to do. The three closest platform rows do not describe a routing policy, a qualification package, or a test.

The Tracker records public disclosure, with a verbatim quote. It is not a sweep of validation files, so it cannot show whether a routing layer was qualified. A thin public record is a disclosure gap. Qualification may still have been done. A validated gateway may still be there. You cannot see either one from these rows.

On the 37 platform and assistant entries, the GxP label is never GxP: 23 are non-GxP, 12 are unclear, and 2 are mixed. Both mixed rows are still at announced stage. One is Bristol Myers Squibb’s Claude agreement. The other is Merck & Co. and Google Cloud (joint release), where the source says the investment “will deploy an agentic platform”. When a row is labeled GxP, the label sits on an application, such as Takeda’s investigation tool, not on these 37. In this public record, the platform is described as ordinary employee work.

Validation follows intended use. That is the principle in FDA’s Computer Software Assurance guidance and the ISPE GAMP AI Guide. It is general guidance, not a finding from the Tracker. Employee chat does not, by itself, put a routing layer on the CSV path. A GxP application that calls the layer for a GxP purpose does.

Jazz Pharmaceuticals, AI Council and Responsible Use Guidelines (2025 report), in production, labeled non-GxP, is a governance disclosure. The Tracker summary says Jazz established an AI Council and AI Responsible Use Guidelines. The source says the guidelines are there “to define expectations and support consistent application”. A council and a guideline are not the routing layer. Jazz is one of the 26 companies with no platform or assistant entry in this reading.

Five questions to ask

The card is the short version. The notes below are what to ask for. They are a checklist for your own layer. They are not findings about any company in the Tracker.

1. Does any GxP work use this layer, and is it on the system inventory? Why it matters. A shared routing layer that a GxP application calls is in scope for CSV, a custom build is likely GAMP Category 5, and if GxP work already uses a layer that is missing from the inventory, write that down first. If everything on it today is non-GxP, make the inventory decision before the first GxP use comes through.

2. Who owns the rule that sends a request to a model? Why it matters. Which model an assistant is sent to, and what the fallback is, changes what the system does, so that rule needs an owner, a version, and a change-control path.

3. Is the list of approved models a controlled record, or a config file? Why it matters. Ask for the risk assessment and the approval record for the models in use now, and for what the control does when a model is taken off the list. None of the Tracker entries above include that record.

4. Would the audit log name the person, the system, and the model that answered? Why it matters. The record should name the person and the system that made the call, name the model version that actually answered after routing and fallback, and be tamper-evident. Field by field: an ALCOA+ audit trail for LLM calls. The wider pre-deployment checklist: what a CSV validator should look for in an LLM gateway.

5. What stops a call to a model that is not approved, and where is the test? Why it matters. Ask for the control that blocks the call, and for a test that was actually run, showing the call was blocked and written down.

In the Pharma AI Tracker, the routing layer is the part these companies have said the least about. A search for the word gateway returns nothing. Three entries describe something like shared access or a control layer, and only Takeda’s summary uses the words “control tower.” If you meet one of these layers, ask which work it serves, whether any of that work is GxP, and what documentation exists.