Model Cards: Documentation, Intended Use, and Limitations

When a language model is trained and deployed, the questions that matter most are rarely about architecture. Practitioners, policymakers, and end users want to know: what was this model trained to do, who should use it, where does it fail, and what risks should I be aware of? These questions are answered by a model card.

A model card is a short document that travels with a machine learning model to communicate the essential facts about it. Google researchers Mitchell et al. introduced the concept in 2018 as a response to a practical problem: models were being shipped without any structured information about their scope, limitations, or appropriate use. The result was models applied in contexts their developers never intended, producing failures that better documentation could have anticipated or prevented.

Model cards borrow their name from trading cards and nutrition labels: the idea that a compact, standardized summary provides enough information to make informed decisions. Just as a food label tells you what is in a product and how much to eat, a model card tells you what a model can do, where it was built to operate, and what goes wrong when you use it outside those bounds. The nutrition label analogy is apt in another respect: just as a label cannot make an unhealthy product healthy, a model card cannot make an unsafe model safe. It can only make the risks legible so that users can make informed choices.

The concept has since become an industry norm. Hugging Face hosts hundreds of thousands of model cards on its model hub. Google, Meta, Anthropic, and OpenAI publish model cards or equivalent documentation for their flagship systems. Several governments and regulatory frameworks now treat model documentation as a compliance requirement rather than a best-practice recommendation. Understanding what goes into a well-constructed model card, and why each section exists, is an essential skill for anyone deploying or evaluating language AI systems.

The timing of the model card movement was not accidental. It emerged alongside a broader shift in how practitioners and regulators thought about AI accountability. As machine learning systems moved from experimental tools into consequential applications, the absence of standardized documentation became increasingly untenable. A hiring algorithm with no documentation of training data composition could not be audited for discrimination. A medical information system with no documented knowledge cutoff gave users no way to know whether its answers reflected current clinical practice. Model cards were designed to close this gap: to create a documentation layer that made previously invisible properties of AI systems legible to the people affected by their deployment decisions.

The problem model cards address is asymmetric information. A model developer knows a great deal about how a model was built: the training data, the evaluation results, the observed failure modes, the population the model was tested against. A downstream user knows almost none of this. Without documentation, users make deployment decisions based on benchmark numbers alone, which are almost always measured in settings that differ materially from real-world deployment.

This mismatch creates several types of harm. A model evaluated entirely on English text may be deployed in a multilingual customer service system, where its accuracy on minority languages is unknown and potentially much lower. A medical information model trained on clinical notes from major academic hospitals may perform poorly on regional dialects, non-standard terminology, or the kinds of informal health questions real patients ask. A toxicity classifier tuned to flag hate speech in social media text may behave erratically on news articles or academic discussions of the same topics.

Model cards make these gaps visible. When a developer writes down what the model was trained on, who it was tested with, and where evaluations showed degraded performance, downstream users can check whether those conditions match their deployment context. This is not a guarantee of safety, but it is a structured way to surface risks that would otherwise remain invisible. The value is epistemic: it shifts ignorance from an accepted default to a documented fact that can be addressed, challenged, or improved upon.

There is also an accountability function. When model behavior causes harm, documentation trails determine whether failures were foreseeable. A company that deployed a hiring algorithm without knowing (or disclosing) that it performed worse on resumes from certain universities faces different accountability than one that documented this limitation and deployed anyway. Model cards create a record of what was known at deployment time. This record matters both for legal purposes and for organizational learning: teams that document failures before deployment are better positioned to learn from them and improve future systems.

A third function is enabling auditing. External auditors, regulators, and researchers who want to evaluate whether a model meets certain standards need documentation as a starting point. Without a model card, auditing a model requires independently discovering the same facts the developer already knows, which is expensive and often incomplete. Model cards lower the barrier to external oversight by providing a structured entry point into the model's characteristics.

Finally, model cards serve an educational function within development teams. Writing a model card forces developers to articulate what they know about their model, including what they do not know. Teams that go through this exercise regularly find gaps in their own evaluation coverage. The discipline of writing the intended use section precisely reveals how often "we'll handle that later" assumptions are embedded in deployment decisions. The limitation section requires acknowledging failures that might otherwise be quietly set aside. In this sense, model cards are as useful for the teams that write them as for the users who read them.

A complete model card covers several distinct categories of information. The sections and their exact naming vary across organizations, but the required content follows a recognizable pattern. The original Mitchell et al. framework proposed nine sections; subsequent implementations have varied this structure while preserving the core content areas. What matters is not which exact sections you include but whether the document covers the key questions a deployer needs answered.

The model details section provides the basic identifying information: the model name and version, the organization that developed it, the release date, the model type (classification, generation, embedding, etc.), and the training framework. This section also documents the license terms and any contact information for questions or reports.

Version numbers matter more than they might appear. Models are frequently updated, fine-tuned, or retrained, and performance can change significantly between versions. A model card without a clear version identifier cannot be compared reliably to evaluation reports, benchmark results, or incident analyses that reference a specific version. Teams that maintain multiple versions of a model need version-specific cards rather than a single shared document, because a limitation documented for version 1.0 may or may not apply to version 1.1.

The license terms in model details are increasingly consequential. Foundation models are released under a variety of license types: fully open-source licenses that permit commercial use, community licenses that prohibit commercial use without additional agreements, research-only licenses, and proprietary licenses with various restrictions. A deployer who does not read the model details section may find that their use case violates the license, creating legal risk that a few minutes of documentation review would have prevented.

The contact information field, often overlooked, serves a practical purpose: it gives users who discover failure modes or security vulnerabilities a channel to report them. Models without contact information create a one-way information flow where developers never learn about problems discovered after release.

The intended use section is where the model developer communicates what the model was designed to do, and equally important, what it was not designed to do. This section is arguably the most important in the card because it establishes the interpretive frame for everything else. Performance numbers are only meaningful relative to a specific use case; limitations are only actionable relative to a specific deployment context.

This section typically covers three elements. The first is the primary intended uses: the specific tasks and contexts the model was built and evaluated for. For a sentiment classifier, this might be analyzing customer reviews in English. For a code completion model, this might be suggesting completions for Python and JavaScript in interactive development environments. For a general-purpose language model, this might be a broader range of text generation tasks with specific caveats about high-stakes applications.

The second element is the intended user population. A medical information model intended for use by trained clinicians has different appropriate-use parameters than one marketed to general consumers, even if the underlying model is identical. The intended user section should specify the professional expertise or knowledge assumed in the user, the institutional context where the model is expected to operate, and whether human oversight was assumed in the design. Models designed with the assumption that a human expert reviews outputs before acting on them are often deployed in contexts where no such review occurs, with predictable consequences.

The third element is explicit statements of out-of-scope uses: use cases the developer has not evaluated and does not recommend. This is often where the most practically important information lives. Uses that are technically possible but not validated, or uses that the developer specifically warns against, deserve explicit documentation rather than the implicit assumption that "anything we didn't mention is fine." Real-world deployment experience shows that users regularly apply models to contexts their developers never anticipated, and explicit out-of-scope documentation creates both a warning and an accountability marker.

The intended use section is not legally binding in most jurisdictions, but it shifts accountability significantly. When a developer explicitly states that a model is not suitable for high-stakes autonomous decision-making and a deployer uses it exactly that way, the deployer bears greater responsibility for the consequences. This accountability function is one reason some developers are careful about how they word intended use statements.

A common failure in this section is vagueness. "Intended for natural language understanding tasks" communicates almost nothing. A well-written intended use section names specific tasks, specific domains, and specific user populations with enough precision that a deployer can check whether their context matches. The test is simple: if a reasonable person could read the intended use statement and deploy the model in a context that causes harm, the statement is not specific enough.

The training data section documents the data used to create the model. This section is critical because training data composition determines almost everything about a model's strengths, weaknesses, and blind spots. A model trained predominantly on formal English text will have fundamentally different characteristics than one trained on multilingual social media text, and those differences matter enormously for deployment decisions.

Complete documentation covers the data sources (web crawls, licensed datasets, proprietary corpora, synthetic data, user-contributed data), the scale in tokens or documents, the time period during which data was collected, the preprocessing pipeline (filtering, deduplication, normalization, toxicity removal), and any known limitations in composition such as demographic skews, domain overrepresentation, or language imbalances.

Data documentation is frequently incomplete in practice. Many large language models are trained on web crawls whose full composition is uncertain: the Common Crawl corpus used to train most foundation models contains billions of documents from sources that are difficult to fully enumerate. The appropriate approach is to document what is known, acknowledge what is uncertain, and flag the implications of uncertainty for downstream use.

One dimension deserving particular attention is temporal coverage. A model trained primarily on data from before a certain date cannot reliably answer questions about events after that date. Models that do not document their knowledge cutoff lead users to expect currency that the model cannot provide. This problem is especially acute for general-purpose language models used for factual question answering, where a knowledge cutoff of even six months can make a material difference to answer quality.

The preprocessing pipeline is often underdocumented. What a developer calls "web-crawled text" might mean raw HTML with minimal filtering, or it might mean aggressively filtered text that removes adult content, low-quality documents, and duplicate content. These decisions have large effects on what the model learns. A model trained on aggressively filtered text will have very different behavior on informal or non-standard inputs than one trained on raw crawl data. Documenting these choices allows downstream users to reason about how the model will behave on their specific data distribution.

Training data documentation also encompasses decisions about data weighting. Foundation models are not typically trained on a uniform sample of the web; training mixes weight different sources differently to improve performance or reduce specific failure modes. When these weighting decisions are undocumented, users cannot reason about which capabilities the training process prioritized or which domains the model may be weaker in despite appearing on the surface to cover them.

The evaluation results section presents how the model performed on benchmarks and evaluations. This section determines how readers interpret the model's quality, and it is where documentation failures most directly mislead users.

A credible evaluation section includes the benchmark performance on standard benchmarks (with the benchmark name, version, and metric clearly identified), documentation of whether evaluation was done on held-out data or a standard test set, disaggregated results broken down by relevant subgroups, and baseline comparisons that give the numbers meaning by showing how the model compares to prior models or simple baselines.

Held-out evaluation and test set contamination deserve particular attention. Evaluation on data drawn from the same distribution as training data inflates performance estimates in ways that do not generalize to deployment. Test set contamination, where training data accidentally includes documents from standard benchmarks, similarly inflates numbers. A model card that documents which evaluation set was used, when it was last updated, and whether any contamination analysis was performed gives readers the information they need to assess how much confidence to place in the reported numbers.

Disaggregated evaluation is the most consequential omission in practice. Overall accuracy numbers hide variation across subgroups. A hiring classifier that achieves 87% overall accuracy might achieve 92% on resumes from male applicants and 79% on resumes from female applicants. The aggregate number looks reasonable; the disaggregated numbers reveal a serious fairness problem. Model cards that report only aggregate metrics give deployers no way to detect this kind of differential performance. The EU AI Act, for example, now requires providers of high-risk AI systems to document performance across relevant demographic groups precisely because aggregate metrics have proven so consistently misleading.

Some model cards separate quantitative analyses from evaluation results to provide more detailed breakdowns. This section might include statistical confidence intervals, performance curves across different operating thresholds, intersectional analysis (performance broken down by combinations of demographic factors), or robustness analyses that show how performance degrades under distribution shift.

The distinction between evaluation results and quantitative analyses is that the latter goes beyond standard benchmark numbers to provide the richer empirical picture needed for high-stakes deployment decisions. The inclusion of confidence intervals, for example, allows users to distinguish a model that reliably achieves 85% accuracy from one that achieved 85% in one evaluation run with high variance. These are very different deployment risks, and aggregate accuracy numbers alone cannot distinguish them.

Intersectional analysis is particularly useful and particularly rare. Performance disparities do not always align with single demographic dimensions. A model might perform similarly for men and women overall, and similarly for different age groups overall, but perform substantially worse for older women specifically. Detecting intersectional disparities requires analysis across combined subgroups, which requires both demographic annotations and enough evaluation data in each subgroup cell to produce reliable estimates. Meeting these requirements is a real constraint in addition to being a documentation challenge.

Threshold analysis is another often-omitted quantitative analysis. For classification models, performance varies with the decision threshold used to convert probability outputs to class labels. A model card that reports accuracy at a single threshold gives deployers no information about the tradeoff between precision and recall at different operating points. Including precision-recall curves or ROC curves allows deployers to set thresholds appropriate to their specific deployment context, where the relative cost of false positives and false negatives may differ substantially from the default.

The ethical considerations section documents the potential harms, biases, and risks associated with the model. This section has a different character than the others: it asks developers to reason about what could go wrong rather than what the model does well. Writing it well requires intellectual honesty that runs counter to the instinct to present a model favorably.

Detailed ethical considerations coverage includes known demographic disparities in model behavior and how they were measured, privacy risks from training data that contained personal information, dual-use potential where the model could be used to generate harmful content or assist with malicious tasks, and environmental impact documentation covering the computational resources used for training and estimated carbon footprint.

The bias and fairness subsection requires the most care. Models trained on historical human-generated text reflect and sometimes amplify historical biases in gender roles, racial associations, cultural stereotypes, and more. The presence of these biases is not primarily a function of developer intent; it is a structural property of training data that reflects the biases present in the culture that produced it. Documenting known biases is not an admission of negligence; it is an acknowledgment of a well-understood property of current training methods.

This section is where model cards most often fail to be useful. A common pattern is a superficial acknowledgment of bias without any specifics: "The model may exhibit biases present in the training data." This statement is technically true of every model trained on human-generated text and provides no actionable information. A useful ethical considerations section names specific known biases, provides evidence (whether from the developer's own evaluations or from external audits), and explains what implications those biases have for intended deployment contexts.

The dual-use section is increasingly important as language model capabilities expand. A model that can be used for legitimate text generation tasks can often be used, with relatively minor modification, for generating disinformation, phishing emails, or content that assists with harmful activities. Developers who have thought carefully about dual-use risks and implemented mitigations should document what they did and what residual risks remain. Developers who have not thought about dual-use risks should say so.

Privacy documentation matters for models trained on real user data or public text that includes personal information. Language models can memorize and reproduce training data, and this memorization creates privacy risks when training data includes names, addresses, medical information, or other sensitive content. Documenting what personal information was in training data and what steps were taken to reduce memorization gives users information they need to assess privacy risks in their specific deployment contexts.

The limitations section translates known failure modes into practical guidance. It answers the question: given everything documented in this card, what should a deployer do?

Effective limitations documentation includes specific types of inputs or contexts where the model performs poorly, behavioral bounds that define what the model will and will not do, the conditions that make documented performance claims valid (and what deviations invalidate those claims), and recommended mitigations such as human review for high-stakes decisions, domain-specific fine-tuning, or restricted deployment scope.

The framing matters here. Limitations stated as abstract properties ("the model may not generalize to out-of-distribution inputs") are less useful than limitations stated as actionable guidance ("this model was evaluated only on formal news text; do not use it for social media analysis without additional evaluation on that domain"). The abstract version is technically accurate but gives a deployer nothing to do. The specific version tells them exactly what to check and what to do if the check fails.

Recommendations derived from limitations should be concrete and deployment-specific. "Use caution in high-stakes settings" is advice that no responsible deployer would disagree with, but it does not tell them what precautions to take. "Apply human review for predictions with confidence scores below 0.75, which reduces the expected error rate from X% to Y%" is advice that can be directly implemented.

Intended use documentation is one of the most consequential parts of a model card because it directly determines how users interpret the model's appropriate scope. Writing it well requires resisting two opposing temptations: the urge to claim broad applicability (which overstates what has been validated) and the urge to disclaim everything (which provides no useful guidance).

The most useful intended use sections are specific about tasks, domains, languages, and user populations. Compare these two formulations:

Weak: "This model is intended for natural language processing applications in customer-facing contexts."

Strong: "This model is intended for sentiment classification of English-language product reviews in e-commerce contexts. It was evaluated on reviews from electronics, apparel, and home goods categories. It has not been evaluated for reviews in non-English languages, reviews of services (rather than products), or social media text."

The second formulation tells a deployer exactly what has been validated and immediately raises the question of whether their context matches. The first formulation appears to cover almost any text-based customer application, which will lead deployers to use it in contexts far outside what was tested.

Specificity also helps when a model is intended for broad use. Rather than claiming broad applicability without evidence, a well-written card documents the breadth of evaluation that supports the broad claim. "This model is intended for general-purpose text generation tasks. It was evaluated across 42 task categories spanning question answering, summarization, code generation, translation, and dialogue, with results reported separately for each category" is both broad and specific. The breadth claim is anchored by the evaluation evidence.

The task of specifying what has been validated rather than what might work requires developers to know their evaluation coverage. Teams that build evaluation infrastructure early in the development process are better positioned to write specific intended use documentation because they have the evaluation data to back their claims. Teams that treat evaluation as a final step before release often find that they have benchmarked aggregate performance but have not measured coverage across the specific task and domain dimensions that intended use documentation requires.

The user population the model was designed and validated for is often as important as the use case. A medical information model designed for use by trained physicians behaves under different appropriate-use constraints than one designed for health-literate consumers, even if the task description is superficially similar. The intended user section should specify the professional expertise or knowledge assumed in the user, the institutional context (clinical setting, educational platform, enterprise software), and whether human oversight was assumed in the design.

Models designed with the assumption that a human expert reviews outputs before acting on them are often deployed in contexts where no such review occurs. Documenting the oversight assumption makes this gap visible. It shifts the responsibility for missing oversight to the deployer rather than the developer, and it creates a record that allows post-incident analysis to identify where the assumption broke down.

Documenting intended users also requires clarity about the distinction between the model's immediate users and its downstream subjects. A hiring algorithm's immediate users are HR professionals; its downstream subjects are job applicants. A content moderation model's immediate users are platform operators; its downstream subjects are the users whose content gets moderated. The interests and vulnerabilities of downstream subjects are often more consequential than those of immediate users, and a complete intended use section acknowledges both.

Language models deployed through APIs create another user layering issue: the model developer's intended users may be developers building applications, but those developers' users may be end consumers with very different backgrounds, expectations, and vulnerabilities. Model cards intended for a developer audience need to be complemented by consumer-facing documentation when the end-user population differs substantially from the developer audience.

The out-of-scope section requires careful work to be useful. Simply listing "anything not mentioned above" is circular. A productive approach is to enumerate the highest-risk use cases that a naive user might attempt, particularly those that have been observed empirically in similar contexts or that follow predictably from the model's capabilities.

Commonly important out-of-scope categories include autonomous high-stakes decisions without human review, applications to populations significantly different from the evaluation population, languages or dialects not included in training or evaluation, and real-time safety-critical applications where the model's error rate would cause harm at deployment scale.

The rationale behind each out-of-scope item is as important as the item itself. "Not suitable for clinical diagnosis" is a useful warning, but "not suitable for clinical diagnosis because the model was trained on de-identified records from academic medical centers and was not evaluated on primary care or community health center data" is substantially more useful. It tells the reader both what to avoid and why, which enables them to reason about analogous cases not explicitly listed.

Temporal scope is often omitted from out-of-scope documentation. A model with a training knowledge cutoff should document that the cutoff exists and which domains are most affected by it. A general language model with a cutoff in early 2024 might still perform acceptably for historical analysis or general reasoning, but would be clearly out of scope for questions about current events, recent legislation, or rapidly evolving technical fields.

Limitation documentation is where model cards most often fall short of their potential. The challenge is converting technical findings into guidance that a non-specialist deployer can act on. Performance metrics mean different things in different contexts, and the gap between what a metric measures and what a deployer needs to know is often larger than it appears.

Every performance metric implies something about deployment risk, but that implication is not self-evident. A 5% error rate on a medical coding task sounds small until you consider that a large hospital processes thousands of claims per day: at that volume, a 5% error rate means hundreds of errors daily. A model card that reports the 5% error rate without contextualizing its implications at realistic deployment scales gives deployers an incomplete picture.

Better documentation translates metrics into deployment implications:

• "The model achieves 95% accuracy on our evaluation set. At a deployment volume of 1,000 queries per day, this implies approximately 50 errors daily. For high-stakes decisions, we recommend human review for cases where confidence scores fall below 0.85, which reduces expected errors to approximately 10 per day."

This kind of quantitative translation requires knowing something about intended deployment scale, which is another reason the intended use section and the limitations section should be written together rather than independently. The intended use section establishes the deployment context; the limitations section should be calibrated to that context. Error rates that are acceptable for a low-stakes recommendation system may be completely unacceptable for an automated administrative decision that affects people's benefits, employment, or legal status.

The deployment volume calculation also reveals something important about model selection. Organizations choosing between models often focus on a few tenths of a percentage point in accuracy. But if Model A achieves 95.0% accuracy and Model B achieves 95.3% accuracy, and both are deployed at scale, the practical difference is small. If Model A has well-documented subgroup performance and Model B does not, the documentation gap is a much more important consideration than the accuracy gap. This reasoning argues for treating documentation quality as a material factor in model selection rather than an administrative checkbox.

Disaggregated performance data is one of the most valuable things a model card can provide, and one of the most commonly omitted. When overall performance looks adequate but subgroup performance is substantially degraded for some populations, aggregate metrics hide a deployment risk.

The challenge with disaggregated evaluation is that it requires demographic annotations in evaluation data, which are expensive to collect, raise their own privacy considerations, and may not be available for all relevant subgroups. There is no simple solution to this challenge. Some relevant subgroups can be identified through linguistic markers (dialect, writing style, vocabulary) rather than demographic annotations. Proxy variables can sometimes capture relevant variation when direct demographic data is unavailable. Adversarial evaluation using inputs constructed to represent specific groups can surface disparities without requiring annotated demographic data.

A model card should be honest about which disaggregated analyses were performed and which were not, rather than presenting aggregate numbers as if they were the complete story. When disaggregated data is unavailable, a responsible card acknowledges this gap explicitly: "We were unable to evaluate performance by demographic subgroup due to the absence of demographic annotations in our evaluation set. Users deploying this model in contexts where demographic equity matters should conduct their own disaggregated evaluation before deployment."

This acknowledgment of uncertainty is not a weakness in the model card; it is information. A deployer who reads this acknowledgment knows that they need to conduct their own evaluation before deploying in equity-sensitive contexts. A deployer who reads a model card with no mention of disaggregated evaluation might incorrectly assume the analysis was done and found no problems. Explicit documentation of what was not measured is as important as documentation of what was measured.

Some failure modes are structural and predictable rather than statistical. A model trained on formal text will systematically underperform on informal text. A model with a knowledge cutoff will systematically fail on questions about events after that date. A classification model trained on balanced classes will have poorly calibrated confidence when deployed on a naturally imbalanced distribution.

These structural failure modes deserve their own documentation distinct from aggregate accuracy numbers. They are not random errors distributed uniformly across inputs; they are predictable degradations for specific input types that a careful deployer can identify and mitigate if they know to look for them. A model card that documents structural failures enables deployers to build targeted mitigations: preprocessing to normalize informal text before it reaches a model trained on formal text, explicit fallback behavior for questions that may relate to post-cutoff events, confidence recalibration for deployment on imbalanced distributions.

Adversarial inputs represent a category of known failure mode that is increasingly important to document as language models are deployed in security-sensitive contexts. Prompt injection, jailbreaking, and adversarial suffixes can cause models to behave in ways far outside their documented capabilities. Testing for these failure modes and documenting both the vulnerabilities found and the mitigations applied gives deployers critical information for designing system-level defenses.

The structural nature of many failure modes also implies something about how models should be monitored after deployment. Aggregate accuracy monitoring will not detect systematic degradation on a specific subgroup unless that subgroup is large enough to affect aggregate numbers. Monitoring strategies designed around known structural failure modes, where specific input types or subpopulations are tracked separately, provide much earlier warning of deployment problems than generic aggregate monitoring.

Model cards did not develop in isolation. They have become increasingly intertwined with regulatory frameworks governing AI deployment, and understanding this context is important for practitioners who need to produce documentation that satisfies internal standards and external requirements.

The EU AI Act, which entered into force in 2024, establishes documentation requirements for AI systems based on their risk category. High-risk AI systems (including systems used in employment, education, essential services, law enforcement, and credit decisions) must be accompanied by technical documentation that covers training data, development methodology, intended purpose, performance characteristics including accuracy metrics, and human oversight provisions. Model cards, if written to cover these areas with sufficient depth, can serve as the core of this required documentation.

In the United States, the NIST AI Risk Management Framework provides a voluntary but influential structure for AI risk management that includes documentation requirements. Several executive orders and federal agency guidance documents have endorsed the NIST framework, making model card-style documentation a de facto expectation for government contractors and agencies deploying AI systems.

Canada, Japan, South Korea, and the United Kingdom have all published AI governance frameworks that include documentation requirements. The common thread across these frameworks is the expectation that developers can articulate, in writing, what a system was designed to do, how its performance was evaluated, what its known limitations are, and who is responsible for it. Model cards are designed to answer exactly these questions.

Regulatory requirements create both an incentive and a risk for model card documentation. The incentive is clear: documentation that satisfies regulatory requirements reduces compliance risk. The risk is that regulatory compliance becomes the goal rather than accountability. A model card written to check boxes rather than inform deployers satisfies neither goal well. The most effective approach is to write model cards that serve deployers, because documentation written for practical use will also satisfy well-designed regulatory requirements.

The model card framework was originally designed for relatively narrow, task-specific models. Applying it to modern foundation models that exhibit broad capabilities across thousands of tasks creates novel challenges. A general-purpose language model cannot have its intended use documented as specifically as a sentiment classifier; its evaluation cannot cover every task it might be applied to; its limitations cannot be fully enumerated without an effectively unlimited evaluation budget.

Several approaches have emerged to address this scale mismatch. One is the use of capability evaluations that characterize what a foundation model can and cannot do across a structured taxonomy of tasks, rather than listing specific intended uses. Instead of saying "intended for question answering about cooking," a foundation model card might document performance across a taxonomy of task types, allowing deployers to reason about where their specific use case falls.

Another approach is layered documentation, where a base model card documents the pre-trained model's characteristics and separate fine-tuning cards document specific deployment versions. This layering reflects the development process: a foundation model is first trained, then fine-tuned for specific tasks, and each stage can introduce or remove capabilities and limitations. A fine-tuned model's behavior can differ materially from the base model, and documentation that treats them as a single artifact will be inaccurate for at least one of them.

Red-teaming reports have emerged as a complement to traditional model cards for frontier models. Red-teaming involves deliberately trying to elicit harmful behaviors from a model before release, documenting what was found, and reporting what mitigations were applied. These reports address dual-use concerns and safety properties in more depth than traditional model card ethical considerations sections, and some developers now publish red-teaming reports alongside or as part of their model cards.

The challenge of documenting emergent capabilities is particularly acute. Large language models exhibit behaviors that emerge at scale and were not present in smaller versions of the same architecture. Some of these emergent behaviors are beneficial (in-context learning, chain-of-thought reasoning); others are concerning (increased capability for generating convincing disinformation at scale). Documenting emergent capabilities requires evaluation of the actual deployed model rather than extrapolation from smaller scale results, and the pace of capability development often makes documentation lag behind the actual capability state.

Several patterns separate model cards that support responsible deployment from those that function as compliance theater. The gap between a useful model card and a superficially complete one determines whether the document influences deployment decisions.

Every claim in a model card should be backed by specific evidence. "The model performs well on standard benchmarks" is not a useful claim. "The model achieves 89.2% F1 on the CoNLL-2003 NER test set, compared to a BERT-base baseline of 84.6%" is a useful claim. The specificity matters because it allows reproducibility, comparison, and challenge.

Evidence-backed claims also create accountability. When a model card says "we found no significant performance disparities across demographic groups," that claim invites scrutiny of how the evaluation was conducted, what groups were included, and what statistical threshold was used to define "significant." This scrutiny is healthy: it surfaces methodological gaps and creates pressure for more rigorous documentation over time. The alternative, vague positive claims that are too imprecise to be challenged, creates a documentation ecosystem where cards signal quality without supporting it.

Specificity requires work. It requires developers to run evaluations with specific, version-pinned benchmark datasets and record the results carefully. It requires demographic annotation on evaluation data. It requires statistical analysis of subgroup performance rather than reporting only the aggregate. Organizations that build this evaluation infrastructure as a core part of their development process rather than a final-stage documentation exercise produce substantially more useful model cards at lower marginal cost.

A model card that only reports successes is not a complete document. Documented failures, limitations, and subgroup performance gaps are among the most useful contents of a model card because they are the parts most likely to prevent deployment mistakes. Developers should resist the instinct to omit or downplay negative results; the whole purpose of a model card is to make known limitations visible before deployment rather than after.

Negative result documentation also has a signaling function. A model card that documents known failures is more credible overall than one that presents uniformly positive results. Sophisticated users learn to be suspicious of model cards with no documented limitations. Any model trained on imperfect data and evaluated on benchmarks that do not perfectly match deployment conditions will have limitations; a card that claims none is either incomplete or misleading. Documented limitations signal that the developer invested in evaluation and documentation quality.

The documentation of negative results also allows cumulative learning across the field. When developers document failure modes specifically and with evidence, other developers can learn from those findings and design their own systems to avoid the same failures. When failure modes are suppressed in documentation, the same failures recur across different development teams who do not know to look for them.

Models evolve. Fine-tuning a pre-trained model changes its behavior in ways that may not be fully characterized by the original model card. Updating a training dataset, even with additional data of the same general type, can shift performance across subgroups in ways that are not easily predicted. Version-specific model cards that track these changes are significantly more useful than cards written at initial release and never updated.

The practical challenge is that updating model cards requires evaluation infrastructure that can be re-run quickly after model updates, and organizational processes that treat documentation updates as part of the release process rather than as an afterthought. This is a tooling and process problem as much as a documentation problem. Teams that automate their evaluation pipelines so that evaluation re-runs automatically after any model update can generate the evidence for updated model cards with minimal additional effort. Teams that treat evaluation as a manual process run before major releases lack the infrastructure to keep documentation current.

Some developers have moved toward continuous documentation, where model cards are updated alongside the model in a version-controlled system. This approach makes it easy to compare documentation across versions and to identify when a specific change introduced or removed a documented capability or limitation. It also creates a audit trail that is valuable for regulatory compliance and incident analysis.

Model cards are not purely developer-produced documents. After deployment, users observe failure modes that were not visible in pre-deployment evaluation. Community bug reports, external audits, and academic research often surface limitations that the original developers missed. Leading model card implementations include mechanisms for acknowledging these post-deployment findings, either by updating the card itself or by linking to external analyses.

The Hugging Face model hub enables community members to flag issues on model cards. Some developers treat their model card as a living document that incorporates findings from the user community. This approach produces substantially more complete limitation documentation than developer-only cards because it benefits from observations across a much wider range of deployment contexts. The accumulated experience of thousands of users applying a model to real-world tasks surfaces failure modes that no pre-deployment evaluation budget could have covered.

Community feedback integration also creates a mechanism for the research community to contribute. When academic researchers conduct audits of commercial models and publish their findings, developers who integrate those findings into their model cards demonstrate a commitment to honest documentation that goes beyond what they discovered themselves. This integration requires organizational openness to external criticism, which is not always easy to maintain when findings are unflattering.

The practical implementation of feedback integration requires clear channels for submitting feedback, someone responsible for reviewing and triaging incoming reports, and a process for incorporating confirmed findings into official documentation. These are organizational requirements, not technical ones. Organizations that do this well treat feedback integration as an assigned responsibility rather than a nominal gesture.

Let's build a practical model card for a text classification model. We'll create structured documentation, perform the kinds of evaluations a model card requires, and generate the disaggregated performance data needed to support deployment decisions.

We start by training a simple sentiment classifier to use as our example model. The code creates a synthetic dataset with demographic metadata attached, which allows us to demonstrate disaggregated evaluation. In a real workflow, this metadata would come from the data collection process.

The gap analysis visualization makes subgroup disparities immediately legible. A deployer reviewing this chart can see at a glance whether any metric exceeds a concern threshold, and use that information to decide whether additional evaluation or mitigation is required before deployment. The choice of a 5% concern threshold is a policy decision, not a mathematical one. Different deployment contexts may set this threshold higher or lower depending on the consequences of differential performance.

Several tools have emerged to support model card creation and maintenance, and practitioners benefit from knowing what is available rather than building documentation workflows from scratch.

The Hugging Face huggingface_hub library provides a ModelCard class that structures model card content and integrates directly with the Hugging Face model hub. Cards on the hub follow a YAML frontmatter convention for machine-readable metadata, with free-text Markdown sections for human-readable content. This dual structure allows automatic aggregation of metadata across models while preserving the narrative documentation that humans need.

Google's Model Card Toolkit, released as an open-source library, provides a programmatic interface for generating model cards from evaluation results. The toolkit connects to TensorFlow Extended (TFX) pipelines and can automatically populate evaluation sections from pipeline artifacts. This automation is significant because it reduces the friction of keeping evaluation results in model cards current after model updates.

Anthropic's model cards for Claude models follow a format that emphasizes safety evaluations alongside standard capability benchmarks, showing the particular importance of safety documentation for frontier models. The structure demonstrates how model card conventions can be adapted for domains where standard ML benchmarks do not fully characterize the relevant risks.

The Model Card Credit Scoring Card (MC3) initiative and similar domain-specific frameworks show how general model card templates can be adapted for regulated industries where specific documentation requirements apply. Financial services regulators, healthcare regulators, and employment law frameworks each impose their own requirements on AI documentation, and domain-specific model card templates help practitioners satisfy those requirements without reinventing the structure from scratch.

Datasheets for Datasets, introduced by Gebru et al. alongside the model card work, extends the model card concept to training data. Just as a model card documents a model's characteristics, a datasheet documents a dataset's characteristics: its composition, collection process, intended uses, and known limitations. In a complete documentation ecosystem, model cards and datasheets reference each other, creating a traceable link from training data properties to model behavior.

Model cards are not a solution to the problems they document; they are a structured way to communicate those problems so that deployers can address them. Several practical challenges limit how effective model cards are in practice.

The first challenge is the documentation burden. A thorough model card requires disaggregated evaluation data, which in turn requires demographic annotations on evaluation sets. These annotations are expensive, raise privacy considerations, and may not be available for all relevant subgroups. Large language models are especially difficult to document fully because their capabilities span so many tasks that no fixed evaluation suite can characterize them. The evaluation budget required to produce a complete model card for a frontier language model would dwarf the budget typically allocated for documentation.

The second challenge is the incentive structure. Model cards are most useful when they document failures, but developers face commercial incentives to present models favorably. Incomplete documentation is not always dishonest; some limitations are not discovered until deployment. But the asymmetry between the costs of negative documentation (reduced perceived quality) and the benefits (reduced downstream harm) creates structural pressure toward incomplete cards. This pressure is particularly acute in competitive markets where benchmark performance is a primary sales metric and documented limitations are treated as weaknesses rather than evidence of rigor.

The third challenge is the reader problem. Model cards are often written by ML engineers and read by procurement officers, policymakers, or product managers who may lack the technical background to interpret performance metrics correctly. A model card that accurately states "95% accuracy on held-out test set" requires the reader to understand what held-out evaluation means, what the test set composition was, and why held-out accuracy may not predict deployment performance. Without that background, the number is easily misinterpreted. Bridging this communication gap without either oversimplifying (losing accuracy) or over-explaining (losing accessibility) is a writing challenge that most model card authors have not been trained for.

The fourth challenge is that model cards describe static snapshots of dynamic systems. A model deployed in a new context accumulates real-world usage that reveals limitations not visible in pre-deployment evaluation. A card written at release does not update itself as these limitations emerge. Maintaining model cards as living documents requires organizational processes and tooling that most teams have not built. The gap between initial release documentation and actual deployed system behavior tends to grow over time as the model is adapted, fine-tuned, and applied to increasingly diverse contexts.

A fifth challenge is the verification gap. Model cards are self-reported documents. There is no standard mechanism for verifying that the evaluation results reported in a model card are accurate, that the training data description is complete, or that the documented limitations are the complete set of known problems. External audits can partially address this verification problem, but auditing is expensive and typically occurs only for high-profile or high-stakes deployments. For the vast majority of models, model card contents are taken on trust. As regulatory requirements for AI documentation tighten, the demand for verifiable documentation will likely increase, but the infrastructure to support verification at scale does not yet exist.

Despite these limitations, model cards represent a meaningful advance over the pre-2018 norm of no structured documentation. They create a vocabulary for discussing model capabilities and limitations, establish accountability through documented deployment decisions, and provide a framework that regulatory requirements are increasingly built around. As documentation tooling matures and evaluation infrastructure becomes more automated, the practical barriers to high-quality model cards are likely to decrease. The current state is imperfect, but the direction of travel is toward more complete, more verifiable, and more current documentation as a standard expectation for any model deployed in consequential contexts.

Model cards are structured documents that communicate what a model can do, who it is designed for, where it fails, and what risks it poses. They emerged in 2018 to address asymmetric information between model developers and downstream deployers, and have since become an industry norm and regulatory requirement in many contexts.

The essential sections of a model card are:

• Model details: Identifying information, version, and license
• Intended use: Specific tasks, user populations, and explicit out-of-scope applications
• Training data: Sources, size, time period, and known limitations
• Evaluation results: Benchmark performance, including disaggregated results by subgroup
• Quantitative analyses: Confidence intervals, threshold curves, and intersectional breakdowns
• Ethical considerations: Known biases, privacy risks, and dual-use potential
• Limitations and recommendations: Actionable guidance derived from known failure modes

The most impactful sections are also the most often neglected: disaggregated evaluation results and honest limitation documentation. Aggregate performance numbers look good but hide subgroup disparities; limitation sections that acknowledge known failures are more useful and more credible than cards that present only successes.

Writing effective model cards requires specificity over breadth in intended use statements, evidence-backed claims in evaluation results, and translation of performance metrics into deployment implications. For foundation models with broad capabilities, documenting intended use requires capability taxonomies rather than specific task lists, and layered documentation that distinguishes pre-trained model properties from fine-tuned deployment properties. The regulatory context makes documentation quality increasingly consequential: frameworks like the EU AI Act treat model documentation as a compliance requirement, and documentation gaps create legal as well as operational risk.

Keeping cards current as models evolve requires automated evaluation infrastructure and organizational processes that treat documentation updates as part of the release cycle rather than an afterthought. Community feedback integration, where post-deployment observations from users and external auditors update official documentation, produces substantially more complete limitation coverage than developer-only cards. Model cards are not a complete solution to deployment risks, but they are an essential tool for making those risks visible before deployment rather than after.

Ready to test your understanding? Take this quick quiz to reinforce what you've learned about model cards.