A smaller benchmark,
with the original holdout.

A pretrained model brings its training history with it. Let the saved split choose which cases to evaluate.

Choose an evaluated selectionTutorial re-splitSaved checkpoint holdout

Same Small checkpoint. Same saved normalizer. Each tile is an actual evaluated scenario ID in the first 1,000-case data slice.

Loading recorded results…

Amber / train: previously in training. Teal / held out: in the checkpoint’s original test split.

Native evaluator results
Measurement	Selected cases

The samples differ. Generator error rises on the saved holdout while objective gap falls. These are observations for two selections; the paper’s full test set and causal memorization effects are separate questions.

Recorded CPU output, 21 October 2026. Changing this selector displays saved results; inference runs in the Python workflow.

Keep the saved membership

The local intersection gives 797 training, 110 validation and 93 test scenarios. Download all three lists, or use the workflow’s ready-to-load Torch split files.

Download the split JSON

Read both complete native receipts

Let the workflow do the rest

Fork the repository and run Pinned CPU holdout comparison from Actions. It prepares the small data slice, selects saved membership, evaluates and checks the exact IDs.

python compare.py \
  --graphkit /path/to/pinned/gridfm-graphkit \
  --work /path/to/temp/comparison

Installation and scope