Model Cards › What a Model Card Contains

A complete model card covers several distinct categories of information. The sections and their exact naming vary across organizations, but the required content follows a recognizable pattern. The original Mitchell et al. framework proposed nine sections; subsequent implementations have varied this structure while preserving the core content areas. What matters is not which exact sections you include but whether the document covers the key questions a deployer needs answered.

The model details section provides the basic identifying information: the model name and version, the organization that developed it, the release date, the model type (classification, generation, embedding, etc.), and the training framework. This section also documents the license terms and any contact information for questions or reports.

Version numbers matter more than they might appear. Models are frequently updated, fine-tuned, or retrained, and performance can change significantly between versions. A model card without a clear version identifier cannot be compared reliably to evaluation reports, benchmark results, or incident analyses that reference a specific version. Teams that maintain multiple versions of a model need version-specific cards rather than a single shared document, because a limitation documented for version 1.0 may or may not apply to version 1.1.

The license terms in model details are increasingly consequential. Foundation models are released under a variety of license types: fully open-source licenses that permit commercial use, community licenses that prohibit commercial use without additional agreements, research-only licenses, and proprietary licenses with various restrictions. A deployer who does not read the model details section may find that their use case violates the license, creating legal risk that a few minutes of documentation review would have prevented.

The contact information field, often overlooked, serves a practical purpose: it gives users who discover failure modes or security vulnerabilities a channel to report them. Models without contact information create a one-way information flow where developers never learn about problems discovered after release.

The intended use section is where the model developer communicates what the model was designed to do, and equally important, what it was not designed to do. This section is arguably the most important in the card because it establishes the interpretive frame for everything else. Performance numbers are only meaningful relative to a specific use case; limitations are only actionable relative to a specific deployment context.

This section typically covers three elements. The first is the primary intended uses: the specific tasks and contexts the model was built and evaluated for. For a sentiment classifier, this might be analyzing customer reviews in English. For a code completion model, this might be suggesting completions for Python and JavaScript in interactive development environments. For a general-purpose language model, this might be a broader range of text generation tasks with specific caveats about high-stakes applications.

The second element is the intended user population. A medical information model intended for use by trained clinicians has different appropriate-use parameters than one marketed to general consumers, even if the underlying model is identical. The intended user section should specify the professional expertise or knowledge assumed in the user, the institutional context where the model is expected to operate, and whether human oversight was assumed in the design. Models designed with the assumption that a human expert reviews outputs before acting on them are often deployed in contexts where no such review occurs, with predictable consequences.

The third element is explicit statements of out-of-scope uses: use cases the developer has not evaluated and does not recommend. This is often where the most practically important information lives. Uses that are technically possible but not validated, or uses that the developer specifically warns against, deserve explicit documentation rather than the implicit assumption that "anything we didn't mention is fine." Real-world deployment experience shows that users regularly apply models to contexts their developers never anticipated, and explicit out-of-scope documentation creates both a warning and an accountability marker.

The intended use section is not legally binding in most jurisdictions, but it shifts accountability significantly. When a developer explicitly states that a model is not suitable for high-stakes autonomous decision-making and a deployer uses it exactly that way, the deployer bears greater responsibility for the consequences. This accountability function is one reason some developers are careful about how they word intended use statements.

A common failure in this section is vagueness. "Intended for natural language understanding tasks" communicates almost nothing. A well-written intended use section names specific tasks, specific domains, and specific user populations with enough precision that a deployer can check whether their context matches. The test is simple: if a reasonable person could read the intended use statement and deploy the model in a context that causes harm, the statement is not specific enough.

The training data section documents the data used to create the model. This section is critical because training data composition determines almost everything about a model's strengths, weaknesses, and blind spots. A model trained predominantly on formal English text will have fundamentally different characteristics than one trained on multilingual social media text, and those differences matter enormously for deployment decisions.

Complete documentation covers the data sources (web crawls, licensed datasets, proprietary corpora, synthetic data, user-contributed data), the scale in tokens or documents, the time period during which data was collected, the preprocessing pipeline (filtering, deduplication, normalization, toxicity removal), and any known limitations in composition such as demographic skews, domain overrepresentation, or language imbalances.

Data documentation is frequently incomplete in practice. Many large language models are trained on web crawls whose full composition is uncertain: the Common Crawl corpus used to train most foundation models contains billions of documents from sources that are difficult to fully enumerate. The appropriate approach is to document what is known, acknowledge what is uncertain, and flag the implications of uncertainty for downstream use.

One dimension deserving particular attention is temporal coverage. A model trained primarily on data from before a certain date cannot reliably answer questions about events after that date. Models that do not document their knowledge cutoff lead users to expect currency that the model cannot provide. This problem is especially acute for general-purpose language models used for factual question answering, where a knowledge cutoff of even six months can make a material difference to answer quality.

The preprocessing pipeline is often underdocumented. What a developer calls "web-crawled text" might mean raw HTML with minimal filtering, or it might mean aggressively filtered text that removes adult content, low-quality documents, and duplicate content. These decisions have large effects on what the model learns. A model trained on aggressively filtered text will have very different behavior on informal or non-standard inputs than one trained on raw crawl data. Documenting these choices allows downstream users to reason about how the model will behave on their specific data distribution.

Training data documentation also encompasses decisions about data weighting. Foundation models are not typically trained on a uniform sample of the web; training mixes weight different sources differently to improve performance or reduce specific failure modes. When these weighting decisions are undocumented, users cannot reason about which capabilities the training process prioritized or which domains the model may be weaker in despite appearing on the surface to cover them.

The evaluation results section presents how the model performed on benchmarks and evaluations. This section determines how readers interpret the model's quality, and it is where documentation failures most directly mislead users.

A credible evaluation section includes the benchmark performance on standard benchmarks (with the benchmark name, version, and metric clearly identified), documentation of whether evaluation was done on held-out data or a standard test set, disaggregated results broken down by relevant subgroups, and baseline comparisons that give the numbers meaning by showing how the model compares to prior models or simple baselines.

Held-out evaluation and test set contamination deserve particular attention. Evaluation on data drawn from the same distribution as training data inflates performance estimates in ways that do not generalize to deployment. Test set contamination, where training data accidentally includes documents from standard benchmarks, similarly inflates numbers. A model card that documents which evaluation set was used, when it was last updated, and whether any contamination analysis was performed gives readers the information they need to assess how much confidence to place in the reported numbers.

Disaggregated evaluation is the most consequential omission in practice. Overall accuracy numbers hide variation across subgroups. A hiring classifier that achieves 87% overall accuracy might achieve 92% on resumes from male applicants and 79% on resumes from female applicants. The aggregate number looks reasonable; the disaggregated numbers reveal a serious fairness problem. Model cards that report only aggregate metrics give deployers no way to detect this kind of differential performance. The EU AI Act, for example, now requires providers of high-risk AI systems to document performance across relevant demographic groups precisely because aggregate metrics have proven so consistently misleading.

Some model cards separate quantitative analyses from evaluation results to provide more detailed breakdowns. This section might include statistical confidence intervals, performance curves across different operating thresholds, intersectional analysis (performance broken down by combinations of demographic factors), or robustness analyses that show how performance degrades under distribution shift.

The distinction between evaluation results and quantitative analyses is that the latter goes beyond standard benchmark numbers to provide the richer empirical picture needed for high-stakes deployment decisions. The inclusion of confidence intervals, for example, allows users to distinguish a model that reliably achieves 85% accuracy from one that achieved 85% in one evaluation run with high variance. These are very different deployment risks, and aggregate accuracy numbers alone cannot distinguish them.

Intersectional analysis is particularly useful and particularly rare. Performance disparities do not always align with single demographic dimensions. A model might perform similarly for men and women overall, and similarly for different age groups overall, but perform substantially worse for older women specifically. Detecting intersectional disparities requires analysis across combined subgroups, which requires both demographic annotations and enough evaluation data in each subgroup cell to produce reliable estimates. Meeting these requirements is a real constraint in addition to being a documentation challenge.

Threshold analysis is another often-omitted quantitative analysis. For classification models, performance varies with the decision threshold used to convert probability outputs to class labels. A model card that reports accuracy at a single threshold gives deployers no information about the tradeoff between precision and recall at different operating points. Including precision-recall curves or ROC curves allows deployers to set thresholds appropriate to their specific deployment context, where the relative cost of false positives and false negatives may differ substantially from the default.

The ethical considerations section documents the potential harms, biases, and risks associated with the model. This section has a different character than the others: it asks developers to reason about what could go wrong rather than what the model does well. Writing it well requires intellectual honesty that runs counter to the instinct to present a model favorably.

Detailed ethical considerations coverage includes known demographic disparities in model behavior and how they were measured, privacy risks from training data that contained personal information, dual-use potential where the model could be used to generate harmful content or assist with malicious tasks, and environmental impact documentation covering the computational resources used for training and estimated carbon footprint.

The bias and fairness subsection requires the most care. Models trained on historical human-generated text reflect and sometimes amplify historical biases in gender roles, racial associations, cultural stereotypes, and more. The presence of these biases is not primarily a function of developer intent; it is a structural property of training data that reflects the biases present in the culture that produced it. Documenting known biases is not an admission of negligence; it is an acknowledgment of a well-understood property of current training methods.

This section is where model cards most often fail to be useful. A common pattern is a superficial acknowledgment of bias without any specifics: "The model may exhibit biases present in the training data." This statement is technically true of every model trained on human-generated text and provides no actionable information. A useful ethical considerations section names specific known biases, provides evidence (whether from the developer's own evaluations or from external audits), and explains what implications those biases have for intended deployment contexts.

The dual-use section is increasingly important as language model capabilities expand. A model that can be used for legitimate text generation tasks can often be used, with relatively minor modification, for generating disinformation, phishing emails, or content that assists with harmful activities. Developers who have thought carefully about dual-use risks and implemented mitigations should document what they did and what residual risks remain. Developers who have not thought about dual-use risks should say so.

Privacy documentation matters for models trained on real user data or public text that includes personal information. Language models can memorize and reproduce training data, and this memorization creates privacy risks when training data includes names, addresses, medical information, or other sensitive content. Documenting what personal information was in training data and what steps were taken to reduce memorization gives users information they need to assess privacy risks in their specific deployment contexts.

The limitations section translates known failure modes into practical guidance. It answers the question: given everything documented in this card, what should a deployer do?

Effective limitations documentation includes specific types of inputs or contexts where the model performs poorly, behavioral bounds that define what the model will and will not do, the conditions that make documented performance claims valid (and what deviations invalidate those claims), and recommended mitigations such as human review for high-stakes decisions, domain-specific fine-tuning, or restricted deployment scope.

The framing matters here. Limitations stated as abstract properties ("the model may not generalize to out-of-distribution inputs") are less useful than limitations stated as actionable guidance ("this model was evaluated only on formal news text; do not use it for social media analysis without additional evaluation on that domain"). The abstract version is technically accurate but gives a deployer nothing to do. The specific version tells them exactly what to check and what to do if the check fails.

Recommendations derived from limitations should be concrete and deployment-specific. "Use caution in high-stakes settings" is advice that no responsible deployer would disagree with, but it does not tell them what precautions to take. "Apply human review for predictions with confidence scores below 0.75, which reduces the expected error rate from X% to Y%" is advice that can be directly implemented.