OPEN STUDY / 11

A case-and-evidence lab

Distinguish records, interpretations, evaluation and retrospective narrative.

About 3 min
Study notes & citation

A case-and-evidence lab. Institute of Mantic Arts. https://www.manticarts.com/en/learning/evidence-lab

STUDY / OBJECTIVE

Set prospective criteria for cases and retain failures and missing observations.

01 / STUDY

Fix the question and outcome definition

Before recording a claim, specify the subject, deadline, outcome space and scoring rule. “Something will change” is difficult to score; a predefined event within a fixed period is more inspectable. Use fictional material for practice rather than collecting sensitive personal information.

02 / STUDY

Retain all outcomes

Keep matches, non-matches, unscorable cases and missing observations. Missingness is neither success nor a reason for silent deletion. Report raw counts and define the denominator. Preserve the original plan when rules change, documenting the reason and separating later exploration from planned analysis.

03 / STUDY

Specify a comparison baseline

A common event can be guessed correctly often. Evaluation therefore needs a baseline, such as always choosing the most frequent outcome. Sample selection, size, information leakage and retrospective rewording affect interpretation. A small teaching table demonstrates documentation, not validation of a mantic system.

WORKED EXAMPLE

Reporting ten fictional records

Ten fictional records contain six matches, two non-matches and two missing outcomes. This demonstrates counting, not an experiment on any system.

  1. Report all categories—6, 2 and 2—out of ten, without dropping failures.

  2. Among eight observed outcomes, six match: 6/8. State separately that two outcomes are missing.

  3. If missing cases are treated first as all non-matches and then as all matches, the total proportion ranges from 6/10 to 8/10. Their actual outcomes remain unknown.

Counts, denominator and missingness are transparent; these proportions have not been compared with a baseline, sampling uncertainty or independent data.

PRACTICE / REFLECTION

Exercises and answers

01Can ten selected successful cases establish a success rate?

Read the explanation

No. The number of all attempts and selection procedure are missing. A complete sample frame, failures, missingness and criteria are needed to define an interpretable proportion.

02How should a claim refined after seeing the outcome be classified?

Read the explanation

Preserve the original and revision time, labeling the revision as retrospective interpretation or exploration. Do not score it as a prior prediction.

A distinction to retain

Preregistration improves transparency but does not guarantee a sound design or correct conclusion; scoring rules also require scrutiny.

Unit output

A prospective scoring plan, an all-outcome table of ten fictional cases and a baseline-and-limitations statement.

REFERENCES

Reading references

References reviewed: 2026-10-24. Examples, record templates and explanations are editorial teaching material. Fictional inputs are not historical events or efficacy evidence. Citations imply no affiliation.

Personal study record; not a grade, assessment or credential.

Connected study paths

Fate systems: structure and interpretation

Editorial study material · 2026-10-24