Responsible Deployment › Access Control
Language AI HandbookAll chapters Responsible Deployment › Access ControlOne section · Sections in this chapter4 / 10 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Who can use a model, and what can they do with it? Access control is the set of mechanisms that answer these questions, from coarse-grained decisions about who gets credentials to fine-grained rules about what queries are permitted. The mechanisms operate at multiple levels and need to be designed together as a system rather than as independent components.
Authentication and Authorization Tiers
At the broadest level, access control begins with authentication (verifying that a user is who they claim to be) and authorization (determining what that verified user is allowed to do). These are conceptually distinct even though they are often implemented together.
Most production language model deployments use API keys as the primary authentication mechanism. An API key is a cryptographic token that a user attaches to each request to identify themselves. Key management systems track which keys belong to which accounts, what usage limits apply to each key, and which capabilities are granted.
An API key is a unique secret token that grants programmatic access to a service. It identifies the caller, allows the service to enforce per-caller rate limits and permissions, and provides an audit trail of usage. API keys should be treated like passwords: kept private, rotated regularly, and revoked immediately if compromised. Unlike passwords, API keys are often sent with every request, which makes their exposure surface broader.
Authorization tiers allow different users to have different levels of access to the same underlying model. Common patterns include:
- Public tier: Heavily rate-limited access to a safety-filtered version of the model, available to anyone who creates an account.
- Standard tier: Higher rate limits, access to more capable model versions, standard terms of service.
- Enterprise tier: High rate limits, access to fine-tuned or specialized model versions, contractual terms including security and privacy requirements.
- Research tier: Special access for academic researchers, often including reduced restrictions in exchange for feedback and research collaboration.
The tiering decision is partly commercial (different price points for different value levels) but also safety-relevant. Higher tiers typically involve more explicit agreement to usage policies and more accountability when violations occur. Enterprise customers sign contracts with liability provisions that provide a legal mechanism for enforcement. Research tier users may have institutional accountability structures that informal users lack. These differences in accountability change the risk calculus for what access levels are appropriate.
Tiering also enables progressive trust building. A new API consumer with no track record starts at a more restricted tier. As they demonstrate legitimate use patterns and accumulate a history, they can apply for higher-tier access. This is analogous to credit scoring: restrictions are proportional to the uncertainty about the user's intentions and behavior, not to assumptions of bad intent.
Usage Policies and Prohibited Use Cases
Access control is not only technical. Usage policies define the terms under which access is granted and the conditions under which it may be revoked. The technical systems enforce the policies, but the policies themselves are normative documents that require careful design.
A responsible usage policy should enumerate prohibited use cases clearly and specifically. Vague prohibitions like "do not use for harmful purposes" are difficult to enforce and create uncertainty about what is allowed. Clear enumeration, for example "do not use to generate non-consensual intimate imagery of real individuals," tells users exactly what is prohibited and gives enforcement systems a concrete target to monitor for.
A good usage policy also distinguishes between prohibited model outputs (content the model should not produce) and prohibited deployment contexts (applications the API should not be used to build). A model that could be used to automate high-volume scam communications should not be used for that purpose even if each individual output is technically innocuous. The prohibition is on the application, not the output. These two categories of prohibition require different enforcement mechanisms: output restrictions can be implemented in content filters, while deployment context restrictions require periodic auditing of how the API is being used.
Usage policies also specify enforcement consequences, including what evidence triggers policy violations and what happens when violations are detected. Staged enforcement, where first violations receive a warning, repeat violations result in tier downgrade, and serious violations result in immediate termination, is more sustainable than immediate termination for everything because it provides a clear signal to users and creates a proportional response to the severity of violations.
The challenge is that usage policies exist on paper but actual enforcement requires monitoring and review capacity that is often limited. Automated systems can detect some policy violations at scale but miss subtle ones. Human review is higher quality but cannot scale to the volume of API traffic in large deployments. The combination of automated pre-screening and sampled human review is the standard approach, but it leaves a residual enforcement gap that organizations should be honest about.
Rate Limiting
Rate limiting controls how many requests a given user or account can make in a given time window. It serves several purposes simultaneously: resource allocation (preventing any single user from consuming disproportionate compute), abuse prevention (making it more costly to use the API for mass harmful content generation), and cost control (for billing purposes).
The sliding window rate limiter tracks requests within a moving time interval. For a limit of requests per window of seconds, the limiter maintains a sorted list of request timestamps. At each incoming request at time , it removes all timestamps older than from the list, then checks whether the number of remaining timestamps is less than :
where:
- is the current request timestamp
- is the window size in seconds
- is the maximum allowed requests per window
- ranges over all previous request timestamps
- denotes the cardinality of the set (number of elements)
If the request is allowed, the current timestamp is appended to the list. The remaining capacity at time is:
This formula gives the exact number of additional requests permitted within the current window, which is useful for returning in API response headers so that clients can implement backoff logic.
The key design decisions in a rate limit system are:
- Window type: Fixed windows reset at regular intervals (e.g., 1,000 requests per hour, resetting on the hour). Sliding windows count requests in the most recent period (e.g., 1,000 requests in the past 60 minutes, continuously updated). Sliding windows are smoother but more expensive to implement, because they require storing individual timestamps rather than a single counter.
- Granularity: Rate limits can apply per API key, per account, per IP address, per user tier, or per endpoint. Finer granularity catches more sophisticated abuse patterns but requires more storage and computation.
- Burst allowances: Some deployments allow short bursts above the baseline limit with a token bucket algorithm, where unused capacity accumulates up to a ceiling. This helps legitimate users who have bursty workloads (e.g., running a batch job) without providing significant uplift to automated abusers who consistently operate at high volume.
import time
from collections import deque
class SlidingWindowRateLimiter:
"""
A sliding window rate limiter that tracks request timestamps
and rejects requests that would exceed the limit.
Implements: allowed(t) = |{t_i : t_i >= t - W}| < R
"""
def __init__(self, max_requests: int, window_seconds: int):
self.max_requests = max_requests # R: max requests per window
self.window_seconds = window_seconds # W: window size in seconds
self.request_times: dict[str, deque] = {}
def is_allowed(
self, api_key: str, current_time: float | None = None
) -> tuple[bool, int]:
"""
Check whether a request from api_key is within the rate limit.
Returns (allowed, requests_remaining).
"""
if current_time is None:
current_time = time.time()
if api_key not in self.request_times:
self.request_times[api_key] = deque()
window = self.request_times[api_key]
cutoff = current_time - self.window_seconds
# Remove requests outside the current window: timestamps < t - W
while window and window[0] < cutoff:
window.popleft()
if len(window) >= self.max_requests:
return False, 0
window.append(current_time)
remaining = self.max_requests - len(window)
return True, remainingTime (s) Allowed Remaining ------------------------------------------ 0.0 YES 4 2.0 YES 3 4.0 YES 2 6.0 YES 1 8.0 YES 0 10.0 NO (rate limited) 0 12.0 YES 0 14.0 YES 0
The sliding window rate limiter rejects the 6th request in the initial burst (at t=8s, five requests have occurred within the last 10 seconds). When the window slides forward past the first request at t=10s, capacity is freed and subsequent requests succeed again. This is more fair than a fixed-window approach, which would allow another full burst at exactly the window boundary, creating a pattern where clients can double their effective rate by timing requests around the reset point.
Content Filtering
Content filtering is the layer of control that operates at the level of individual requests and responses rather than at the account level. A content filter inspects the input (user query) and potentially the output (model response) and blocks, modifies, or flags content that violates policy.
Input filters prevent certain types of queries from reaching the model at all. They typically operate by classifier models trained to detect prohibited categories: requests for weapons synthesis, child sexual abuse material, targeted harassment instructions, and similar. Input filtering adds latency but prevents wasted compute on requests that would need to be blocked anyway. More importantly, it prevents the model from ever generating the prohibited output, which matters because the process of generating harmful content may itself have externalities (the generated content could be leaked through logs, for example) even if it is blocked from the user.
Output filters inspect model responses before they reach the user. Because the model may generate prohibited content in response to an innocuous-looking input (through prompt injection, jailbreak attacks, or unexpected model behavior), output filtering provides a second line of defense. Output filtering is more expensive computationally since it operates on the generated text, which is typically longer than input prompts. In streaming deployments, where the response arrives token by token, output filtering requires either buffering the entire response before delivery or running incremental classification as tokens arrive, which is technically more complex.
The tradeoff is that both types of filters have false positive and false negative rates. A filter that is too aggressive will block legitimate requests, frustrating users and reducing the model's utility. A filter that is too permissive will allow prohibited content through. The operating point needs to reflect the consequences of each type of error in the deployment context. Different deployment contexts have different costs for each error type: a children's educational platform should tolerate many false positives to catch the few true positives, while a general developer API might weight false positives more heavily because developer productivity is the core value proposition.
Content filtering is not a static component. As adversaries discover which inputs evade the filter, they adapt. The distribution of attempted harmful inputs shifts toward the filter's blind spots. This cat-and-mouse dynamic means that content filters need continuous updates, which requires a steady pipeline from flagged outputs and incident reports back to classifier retraining. A filter trained once at deployment time will drift from adequate to inadequate without this pipeline.
Human Review and Escalation
Automated content filters and rate limiters are necessary but not sufficient. Some categories of concern require human judgment that classifiers cannot provide, particularly for cases where context matters more than the literal content of a query, where the potential harm is severe enough to warrant careful review, or where the filter's confidence is low.
Human review serves several functions in the access control stack. Sampled review of high-risk flagged outputs calibrates filter performance by providing ground-truth labels. Escalated review of borderline cases provides accountability for edge cases that automated systems cannot confidently classify. Investigative review of suspected policy violations allows enforcement decisions to be made by people with full context rather than by algorithms operating on individual requests.
The organizational capacity question is central: human review does not scale to the volume of requests that large deployments generate. Effective designs use human review sparingly, targeting it at the cases where it is most valuable. Probabilistic sampling across all request types ensures broad coverage. Deterministic escalation for high-severity flags ensures that no automatic decision is made on the most consequential cases. Adversarial probing by internal red teamers provides coverage that organic user traffic never will.