Content quality review

Review MDX documentation for whether it reliably helps people and agents understand, decide, and act. The review uses five diagnostic lens assessments and...

Purpose

Review MDX documentation for whether it reliably helps people and agents understand, decide, and act. The review uses five diagnostic lens assessments and one evidence-backed quality rating. It does not produce criterion scores, a composite score, percentages, or issue-rate calculations.

Rubric version

The current version is 2.0.

This file owns the review lenses, rating rules, MDX-specific expectations, and draft exceptions. Review reporting owns evidence and delivery. The reviewer workflow owns delegated execution.

Review sequence

  1. Read the complete document without consulting previous versions, scores, or reviews.
  2. Establish its core task from the title, description, and body: the main thing it promises to help the reader understand, decide, or do.
  3. Confirm the PM’s mechanical validation result. Keep validation failures separate from content quality unless they also cause a demonstrated reader problem.
  4. Examine the document through all five review lenses.
  5. Record each supported problem once as a finding, noting every lens it affects.
  6. Assess each lens as Strong, Mixed, or Weak from the supported evidence.
  7. Assign the holistic quality rating from the effect of the substantive findings on the core task.

Do not count defects, calculate rates, or force a finding under every lens. A lens with no supported problem needs no finding.

Mechanical validation

Mechanical requirements are gates, not quality findings:

  • title and description must be present and non-empty.
  • MDX tags must balance.
  • required attributes on present content elements must exist.
  • internal links and anchors must resolve where they can be checked locally.

The PM runs node authoring-kit/scripts/check-page-quality.mjs <path.mdx> before review. A failed gate prevents promotion but does not automatically make the content Low quality. Report a separate content finding only when the defect also produces a specific reader consequence.

Content sufficiency

A short page can be excellent when it completely fulfils a narrow purpose. A long page can still be insufficient. Frontmatter, imports, generated token renderers, redirects, and decorative examples do not substitute for guidance.

Record a substantive finding when the body does not contain enough original explanation for the reader to complete the page’s core task without inspecting embedded UI, generated data, or linked pages. If the missing guidance makes the core task unreliable, the page is Low quality. Intentionally navigational pages are complete when they route readers clearly and that is their stated purpose.

Review lenses

The lenses guide inspection; they are not scores. Assess every applicable lens using the same positive direction:

Stored valueDisplay assessmentMeaning
strongStrongNo supported substantive finding affects this lens. Optional polish does not change it.
mixedMixedA supported, contained weakness affects this lens.
weakWeakSupported evidence shows that this aspect of the content is materially unreliable.

Every Mixed or Weak assessment must be supported by at least one substantive finding. Every substantive finding must identify at least one Mixed or Weak lens. Explain the problem once even when it affects several lenses.

Lens assessments diagnose where the document succeeds or needs work. Do not convert them into numbers, average them, weight them, or derive the overall quality rating from them.

A High-quality document has no substantive findings, so all applicable lenses must be Strong. A Medium- or Low-quality document must have at least one Mixed or Weak lens. This is a consistency check, not a formula for choosing between Medium and Low; that boundary still depends on the effect of the evidence on the core task.

Clarity

Can the reader understand each important statement on the first read?

Look for:

  • sentences that cannot be resolved to one meaning;
  • missing referents, circular definitions, fragments, or tangled clauses;
  • contradictions between passages that concern the same property under the same conditions;
  • wording that forces unnecessary re-reading.

A recoverable typo or stylistic preference is not a substantive clarity finding by itself.

Ambiguity control

Store this assessment under the ambiguity key. The display name makes the direction explicit: Strong ambiguity control is desirable.

Could reasonable readers make materially different design, implementation, or writing decisions?

Look for:

  • instructions with more than one consequential reading;
  • missing conditions, boundaries, or exceptions needed to choose an action;
  • conflicting directions;
  • present AI-facing fields that are not self-contained, specific, or useful for choosing and applying the guidance.

Different permitted choices are not automatically ambiguous. Show why the stated task needs a constraint before requesting one. Conventional design language such as “visual noise” or “sparingly” is acceptable unless it produces meaningfully different outcomes in context.

Structure

Can the reader find the guidance they need, in an order that supports the core task?

Look for:

  • related information split across distant sections;
  • the same decision stated in several places, with differences that leave the reader unsure which statement to follow;
  • topics placed before the concepts needed to understand them;
  • headings that obscure rather than expose the page’s decisions;
  • a missing content element when its absence leaves the page’s purpose incomplete.

Do not use slot counts. Judge whether the document contains and organises the material its purpose requires.

Terminology

Are decision-affecting technical or domain terms introduced and used consistently?

A key term has a specialised meaning and would change a reader’s decision if misunderstood. An invented MDS term must be defined. A linked component name is already defined by its destination, and ordinary design vocabulary with one accepted meaning does not need a glossary.

Record a finding when a key term is undefined, changes meaning, or competes with another term for the same concept. Quote every relevant use when inconsistency is the problem.

Actionability

Can the reader apply the guidance without inventing missing policy?

Look for:

  • instructions missing the condition, action, or relevant outcome;
  • steps whose sequence changes the result but is not stated;
  • examples that conflict with the rule they are meant to demonstrate;
  • guidance that describes a topic without helping the reader make the decision the page promises.

Descriptive or intentionally navigational content is not defective merely because it contains no instruction.

MDX content expectations

Use content elements for their intended jobs. Their presence is not a score and their absence is a finding only when it harms the document’s purpose or violates a mechanical requirement.

  • A component page normally needs a useful <WhenToUse> block with directive <Rule> elements. <WhenNotToUse> is needed when adjacent choices would otherwise remain unclear.
  • A reviewable <Example> requires caption and aiCaption. An <Example captionWarning={false}> is decorative and excluded from review.
  • A <Rule> requires a useful aiRule; repeating the visible rule verbatim adds no agent utility.
  • synopsis should identify the primary subject and enough scope to distinguish the page from related pages.
  • keywords and alternativeNames are useful only when the subject genuinely benefits from them. Do not request filler metadata.

AI-facing fields (synopsis, aiRule, and aiCaption) should be self-contained, specific, and useful to an agent. In particular:

  • aiRule states what to do or avoid and the boundary that determines when it applies.
  • aiCaption states the guidance, context, setup, and observable reason or outcome.

When a present field fails these expectations, report the reader or agent consequence through the appropriate lens. Do not count the same field several times.

Quality rating

Apply the first matching condition:

ConditionStored valueDisplay rating
The document does not reliably support its core task because required guidance is absent, a consequential contradiction or ambiguity blocks it, or substantive problems are pervasivelowLow quality
The core task remains possible, but one or more localised substantive findings could cause confusion or inconsistent decisionsmediumMedium quality
No supported substantive finding remains; any suggestions are optional polishhighHigh quality

A substantive finding changes what a reader can understand, decide, or do. Optional polish improves presentation without a demonstrated effect on those outcomes. The rating comes from the findings that are not known: see Known items.

Record qualityReason as a short explanation of the decisive evidence. The overall rating is a holistic judgement about the core task, not a calculation from the lens assessments. Do not derive it from the number of Strong, Mixed, or Weak assessments, the number of findings, document length, mechanical validation, research coverage, or effort needed to fix the page. A serious problem can require a small edit; several local problems can still leave the core task possible.

The rating covers only the reviewed content. It does not prove factual correctness against sources the reviewer could not access, research completeness, or readiness to publish.

Finding standard

Every substantive finding must include:

  • a quote or a precise missing instruction;
  • the affected reader decision or task;
  • the likely consequence;
  • the lens or lenses involved;
  • either a supported edit or one concrete author question.

A possible interpretation without demonstrated consequence belongs under uncertainty. Do not invent policy to make a rewrite more specific. When the intended rule is not established, ask for the decision instead of supplying one.

Use neutral report IDs (F1, F2, and so on). Do not add critical/minor labels or another severity scale. Separate substantive findings from optional polish; lens assessments show where the findings apply, while the quality rating communicates their combined effect on the core task.

Known items

A review context may mark a real problem as known: recorded and accepted for now (see Review context). A known item is reported under its own heading with its backlog ID, not an F-number. It is not a substantive finding: it does not make a lens Mixed or Weak, does not lower the quality rating, and qualityReason states the exclusion. A known item that blocks the core task, or is a false claim, does not qualify: count it as a normal finding.

Coverage

Coverage is a PM assessment, not a reviewer lens or rating input. It compares the completed page with the Decision topics in _ak/<document-key>/research.md.

StatusMeaning
completeEvery relevant research topic has an explicit page position. A deliberate statement that MDS publishes no rule counts as a position.
gapsOne or more relevant topics have no page position. List each topic and cite no passage found.
unassessedNo usable research file exists. State why it could not be assessed.

For every assessed topic, cite the page passage that takes a position. Do not infer topics from the reviewer’s knowledge, calculate a percentage, or turn Coverage into a quality score. A page with gaps needs coverage work before promotion; unassessed needs human verification.

Pipeline draft scaffolding

For files under .authoring-kit-cache/documents/<directory>/<document-key>/drafts/ or ak-<document-key>-best-draft/, ignore scaffolding that disappears during manual promotion:

  • navTitle: Authoring Kit Draft;
  • imageSrc="./image-required.svg" placeholders;
  • a missing imageAlt where the component intentionally uses caption and aiCaption instead.

Judge the guidance, not its temporary packaging. Apply these exceptions only to pipeline drafts, never to live pages.

Review standards

  • Prefer exact, concrete, single-meaning guidance.
  • Preserve established intent in suggested edits.
  • Require a definition only when misunderstanding the term changes a decision.
  • Distinguish an unanswered question from an instruction the page must contain to fulfil its promise.
  • Preserve useful guidance and name specific strengths, not only defects.

See also