Responsible Deployment › The Release Decision
Language AI HandbookAll chapters Responsible Deployment › The Release DecisionOne section · Sections in this chapter2 / 10 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Before any model reaches users, a team must make a foundational judgment: is this model safe enough to release, to which audiences, and under what conditions? This is harder than it sounds. There is no universal threshold for "safe enough," and the right answer depends on the deployment context in ways that cannot be evaluated in the abstract.
The release decision is a recurring judgment, not a one-time gate at the beginning of deployment. It gets revisited whenever the model is updated, whenever the deployment context changes, and whenever monitoring reveals that the initial risk assessment was incomplete. Treating it as a one-time event rather than a continuous responsibility is one of the most common failures in model deployment practice.
Risk Profiling
The starting point is an honest characterization of what the model can do that might cause harm. This is different from capability evaluation for benchmark performance. Risk profiling asks a different set of questions:
- What are the model's most capable behaviors in domains that could be misused, such as code generation, persuasion, biological synthesis routes, or disinformation production?
- What known failure modes does the model exhibit, such as hallucination rates, sycophancy under pressure, or loss of refusal under jailbreak conditions?
- What populations are likely to use this model, and which of those populations are most vulnerable?
- What is the counterfactual access to equivalent capabilities? If similar capabilities are already freely available, the marginal risk of one more release may be low.
Risk profiling cannot be done by a model developer alone. It requires red team exercises where adversarial testers specifically attempt to elicit harmful outputs, domain expert review for high-stakes fields like medicine or law, and structured analysis of deployment scenarios. As the Model Cards chapter established, this profiling work directly feeds the documentation that describes intended use and known limitations.
Red teaming is a structured adversarial evaluation process in which a group of testers attempts to find harmful behaviors in a model by probing it with inputs designed to circumvent safety measures or elicit undesirable outputs. The term comes from military practice, where a "red team" plays the role of an adversary to test defenses. In language model contexts, red teamers explicitly try to elicit dangerous outputs, find jailbreaks, and identify failure modes that normal evaluation would not surface.
The output of risk profiling is not a binary pass/fail verdict. It is a map of the risk space: which behaviors pose which level of risk, under which conditions, to which populations. This map informs every subsequent decision in the deployment process. A team that produces only a verdict ("the model passed") without producing the map has done the minimum work rather than the necessary work.
Risk profiling also has a temporal component. A risk profile that is accurate at the time of initial deployment can become stale as the world changes. When external events make certain topics more salient, when adversaries develop new techniques, or when the model's capability profile shifts through fine-tuning, the risk map needs to be updated. Building a cadence of risk profile review into the deployment lifecycle is as important as producing the initial assessment.
Use Case Scoping
Risk profiling answers what the model can do that is harmful. Use case scoping answers what the model will be used for in this specific deployment, and whether that context changes the risk calculus.
A model released as a general-purpose chatbot faces a much broader risk surface than the same model deployed as a specialized code review assistant inside a software company's internal tooling. The chatbot will receive queries from an unknown population with unknown intentions, including potentially vulnerable users. The code review assistant has a constrained input space, a professional user population, and limited pathways to harm.
Scoping the use case does not just narrow the risk surface. It also changes the evaluation criteria. For the general chatbot, you need to evaluate safety across a broad distribution of possible queries. For the code review assistant, you need to evaluate whether the model handles malicious code correctly, whether it gives security advice that holds up to expert scrutiny, and whether it fails gracefully on edge-case inputs. Different deployments require different safety evaluations, and a safety suite designed for one deployment context will often miss important failure modes in another.
Use case scoping also matters for the system prompt architecture. A model deployed with a carefully designed system prompt that constrains its persona, permitted topics, and response style faces a different risk profile than the same base model deployed without such constraints. Evaluating the base model's safety properties is necessary but not sufficient for evaluating the safety of the deployed system. The system prompt is part of the deployment, and its safety-relevant properties need to be evaluated as part of the whole.
The practical output of use case scoping is a set of deployment constraints: this model may be deployed in contexts A, B, and C, with these restrictions, and not in contexts X, Y, and Z. Those constraints then flow directly into the access control architecture and into the terms of service that govern how the deployment can be used.
The Dual-Use Problem
Many capabilities that make language models useful also make them potentially dangerous. This is the dual-use problem, and it has no clean resolution. Understanding it clearly, however, helps decision-makers avoid both extremes: neither pretending the problem does not exist nor treating it as an insurmountable barrier to any deployment.
A model that can explain chemical reactions clearly enough to help chemistry students also has the capability to explain synthesis routes for hazardous compounds. A model that can write persuasive essays for debate practice can also write disinformation at scale. A model that can summarize medical literature can give medical advice that harms patients who receive it without appropriate clinical context. The same underlying capability serves both beneficial and harmful uses, and no amount of careful deployment architecture fully separates them.
Release decisions must explicitly grapple with this tension rather than treating it as unsolvable. The relevant questions are:
- How much capability uplift does this model provide over freely available alternatives?
- How accessible is the information the model provides through other channels?
- How severe and reversible are potential harms?
- What is the realistic population of users who would use this capability harmfully versus benignly?
None of these questions has an easy quantitative answer, but asking them forces decision-makers to reason explicitly about tradeoffs rather than defaulting to either "capabilities are too important to restrict" or "any risk makes release unacceptable." Both extremes fail to account for the real structure of deployment decisions.
A useful framing is to consider the distribution of intent across the likely user population. For most queries, even queries that touch on sensitive topics, the overwhelming majority of users have benign intentions: curiosity, education, research, creative work, professional tasks. A filter or restriction that blocks all users to prevent the rare malicious user has real costs to the many. But for capabilities where misuse causes severe, irreversible harm and where the benign use case has adequate alternatives, restriction is justified even at the cost of inconvenience to legitimate users. The right operating point depends on the specific harm profile, not on a blanket policy.