Subject Index Evaluation

1. Purpose

This standard defines how IndexPDF evaluates and scores a finished back-of-book subject index against its source publication.

An evaluation under this standard answers four questions:

1. What subjects and reader tasks does the source require the index to support?
2. Are the index's headings and locators supported by the cited pages?
3. Can readers find every important treatment that should be indexed?
4. Does the index work as a coherent navigation system?

The result is not merely a numerical score. A complete result consists of:

• six dimension ratings and a weighted score out of 100;
• item-level findings and explanations;
• uncertainty and audit-completeness information; and
• independent publication-readiness gates.

This document is normative for an evaluation only when the evaluation report explicitly identifies IndexPDF Subject Index Evaluation Standard 0.2.

2. What this standard covers

The standard applies to subject indexes for books and comparable long-form publications. It evaluates the delivered index, regardless of whether the index was created by a person, software, or a combined process.

The default evaluation covers:

• meaningful subject coverage;
• editorial selectivity;
• conceptual and stance fidelity;
• heading and subheading architecture;
• locator existence, scope, treatment, and complete-path fit;
• missing access and reader tasks;
• cross-references and alternate access;
• density, arrangement, mechanics, and whole-index coherence; and
• the validity and completeness of the evaluation evidence.

The default score does not evaluate the candidate creator's internal workflow, AI policy, commercial process, data governance, or release approvals. Digital-link behavior, EPUB semantics, PDF/UA, WCAG conformance, multilingual collation, and other delivery requirements are evaluated only when a named profile is added to the report. Profile results are reported separately from the default subject-index score.

Name, author, Scripture, geographic, cumulative, database, citation, search-ranking, and journal-inclusion indexes require separate or additional profiles.

3. Governing principles

4. Source and audit scope

The normative candidate-quality, locator, architecture, and presentation requirements are published separately as the IndexPDF Subject Indexing Rules 0.2. An evaluation under this standard SHALL apply that immutable edition and cite applicable rule anchors in its findings.

5. Evaluation sequence

A full evaluation follows this sequence:

1. Identify the source and construct the page map.
2. Define and approve source chunks.
3. Freeze the evaluation policy.
4. Discover expected subjects and reader tasks without seeing the candidate.
5. Synthesize and independently review the benchmark.
6. Freeze the benchmark.
7. Mechanically normalize the delivered candidate without editorial repair.
8. Account for every delivered path, locator, range, and cross-reference.
9. Audit each locator assignment against the source.
10. Audit every benchmark treatment for missing access.
11. Audit global structure, cross-references, density, mechanics, and coherence.
12. Calculate the six dimensions deterministically.
13. Produce item explanations, the scorecard, gates, limitations, and the public report.

The benchmark is candidate-blind. Candidate wording or structure may not be used to decide what the source ought to have indexed.

Candidate normalization is mechanical. The evaluator may resolve layout structure, but may not repair, deduplicate, rewrite, or reinterpret the delivered index before scoring it.

6. Locator evaluation

Each assessable locator receives three related but distinct judgments.

6.1 Treatment depth

Treatment depth asks how much independently useful information the destination supplies about the complete heading path.

attribution_only, citation_only, and incidental_example are weak only when the passage supplies no independently useful information about the complete path.

6.2 Complete-path fit

Complete-path fit asks whether the entire heading path accurately describes the treatment.

The displayed locator grade is:

This grade explains the locator's strengths and limitations. It is not itself the locator's rating credit.

6.3 Keep decision and rating credit

A locator may be supported with mixed treatment. For example, a supported locator with mixed treatment and exact fit has a displayed diagnostic grade of 70 and reliability credit of 1. A weak, contentless mention cannot be kept. Strong treatment with a material relationship or stance mismatch cannot be kept as delivered.

7. Page-reference Reliability

This standard uses binary keep precision:

Expected-treatment recall is:

The base Page-reference Reliability rating is:

This balances two risks:

• precision prevents unsupported or materially misdescribed locators from receiving credit; and
• recall prevents a sparse index from scoring highly merely because its few locators are accurate.

Treatment and fit diagnostics remain visible at item level but are not averaged into keep precision.

8. Six scored dimensions

Each dimension is rated on a five-point scale. Dimension points are calculated from the rating and weight, and the six point values sum to a maximum of 100.

8.1 Editorial Selectivity

Editorial Selectivity contributes up to 15 points:

• up to 10 points from treatment selectivity; and
• up to 5 points from density fit.

Treatment-selectivity credit is 1 for substantive treatment, 0.5 for mixed treatment, and 0 for weak or contentless presence. The density component uses two chapter-level measures, weighted equally and aggregated by indexable source word count:

Density is a calibration measure, not a quota. It does not decide which subjects should be indexed, and a threshold alone does not establish overindexing or underindexing.

8.2 Conceptual Fidelity and Mechanics

Conceptual-fidelity and heading-architecture nodes use these credits: pass 1, minor issues 0.85, major issues 0.55, and fail 0.

Mechanics nodes use: pass 1, cosmetic issues 0.95, minor issues 0.85, major issues 0.55, and fail 0.

8.3 Findability

Findability combines:

• reader-task success: 60%;
• heading and access architecture: 30%; and
• cross-reference validity: 10%.

If cross-references are genuinely inapplicable, the first two components are renormalized to two-thirds and one-third.

8.4 Caps and rounding

Dimension ratings are rounded to the nearest half point after applicable caps and uncertainty rules. Editorial Selectivity uses its documented 10-plus-5 point calculation. Weighted points are rounded to two decimal places.

Caps prevent a strong average from concealing high-value omissions, distributed unsupported patterns, major stance failures, reader-task failures, defective access architecture, or systematic mechanical problems.

Item grades are explanatory. They SHALL NOT be averaged to reconstruct a dimension rating or total score.

9. Critical gates

The score and gates are separate results. Gates do not subtract points; they identify failures too serious to be concealed by an average.

A complete public result SHALL report every gate as pass, fail, not applicable, or otherwise explicitly unresolved. A numerical score without gate status is not a complete result under this standard.

10. Audit completeness and uncertainty

Full audit

A full audit accounts for every frozen denominator exactly once. Required locators, treatments, subjects, tasks, references, and structure nodes may not remain not_measured.

Pilot or sampled review

A pilot may be used to calibrate judgments or identify likely problems. It SHALL NOT be represented as a full-index score or full conformance result. Its population, selection method, and unexamined areas must be disclosed.

Uninspectable evidence

uninspectable is used when the destination or evidence cannot be judged reliably. It is not silently counted as success or failure. The calculation reports lower and upper bounds. A score is withheld when uncertainty could change the rounded rating or applicable cap.

11. Evidence and reproducibility

The calculation uses validated structured records. Explanatory prose may clarify a judgment but cannot alter its category or arithmetic.

A reproducible evaluation identifies:

• the exact source and page map;
• the approved source chunks and inclusion policy;
• the frozen evaluation policy;
• the candidate-blind benchmark;
• the normalized candidate and complete item inventory;
• the locator, missing-access, cross-reference, and structure audits;
• the scoring rubric and calculation profile;
• audit mode, uncertainty policy, deviations, and limitations; and
• dimension results, caps, gates, and item explanations.

Hashes join exact records and help detect accidental input mix-ups. They are content identities, not security attestations.

Two evaluations are directly comparable only when their source, benchmark, policy, page map, chunks, inclusion scope, audit mode, uncertainty policy, rubric, and calculation profile are compatible.

12. Public report requirements

A public report under this standard SHALL include:

• candidate and source identification;
• this standard's version;
• scoring-rubric and calculation-profile identities;
• full or pilot audit status;
• all six ratings, weights, and awarded points;
• the total score and uncertainty status;
• all critical gate outcomes;
• raw counts and denominators needed to interpret precision and recall;
• concise item-level explanations and rule identifiers;
• material limitations or deviations; and
• the evaluation date.

Public reports should not reproduce protected source text or expose secrets, private file paths, or storage identifiers. Evidence may be represented by stable internal identifiers and public-safe summaries.

13. How to cite an evaluation

Use this statement for an evaluation that satisfies the requirements above:

This subject index was evaluated under the IndexPDF Subject Index Evaluation Standard 0.2. The evaluation report states its audit mode, source and benchmark identities, score, uncertainty, and critical-gate results.

A shorter website label may be used when it links to the full report and this standard:

Scored under the IndexPDF Subject Index Evaluation Standard 0.2.

14. Authority

The editorial criteria are informed by ANSI/NISO Z39.4-2021, established professional indexing guidance, and applicable technical standards. The normative scoring requirements are the provisions of this document.