# Optical Recognition of Handwritten Digits

`optdigits-3-8.csv` is a mechanically extracted subset of the UCI Optical
Recognition of Handwritten Digits dataset. It contains every official training
and test example labeled 3 or 8, in the original row order:

| Split | Threes | Eights | Total |
|---|---:|---:|---:|
| Training | 389 | 380 | 769 |
| Test | 183 | 174 | 357 |
| Total | 572 | 554 | 1,126 |

Each row has an explicit split, its zero-based row index in the corresponding
official file, the original label, and 64 pixel-block counts in row-major
order. The pixels remain integers from 0 through 16. Runtime code divides them
by 16 and prepends a constant bias coordinate.

## Source and attribution

E. Alpaydin and C. Kaynak, *Optical Recognition of Handwritten Digits*
[Dataset], UCI Machine Learning Repository, 1998.

- Dataset page: https://archive.ics.uci.edu/dataset/80/optical%2Brecognition%2Bof%2Bhandwritten%2Bdigits
- DOI: https://doi.org/10.24432/C50P49
- License: Creative Commons Attribution 4.0 International (CC BY 4.0),
  https://creativecommons.org/licenses/by/4.0/

UCI describes the source as normalized 32-by-32 handwritten-digit bitmaps from
43 writers. Nonoverlapping 4-by-4 blocks were counted to produce 8-by-8 arrays
whose entries range from 0 through 16. Thirty writers supplied the training
set and a different thirteen supplied the test set.

## Reproduction

The official downloaded ZIP used for this extraction had SHA-256

`0d7b054fea010270e9b3f06411c654c5e59547732ad626381980baffe0a23fb0`.

The extracted source files had SHA-256:

- `optdigits.tra`:
  `e1b683cc211604fe8fd8c4417e6a69f31380e0c61d4af22e93cc21e9257ffedd`
- `optdigits.tes`:
  `6ebb3d2fee246a4e99363262ddf8a00a3c41bee6014c373ed9d9216ba7f651b8`

The generated `optdigits-3-8.csv` has SHA-256
`8bfee028e8734f68a76af77fbf7b56ee6cae8dfc2c3aeb3fb8df416968b7cba1`.

Regenerate the subset with:

```sh
node tools/build-ml-dataset.mjs /path/to/optdigits.tra /path/to/optdigits.tes
```

The script validates row widths, integer values, pixel ranges, labels, counts,
and preserves the official split and row indices.

## Original 32-by-32 companion images

`optdigits-3-8-originals.bin` contains the corresponding normalized binary
$32\times32$ bitmaps in exactly the same subset order: 769 training images,
then 357 test images. Pixels are packed eight per byte, after a 20-byte header,
so the complete companion asset is 144,148 bytes. Its SHA-256 is

`258e26665d9a538534a86c0b3223f58b8508f6c6950f422c5dc7537dd55889e9`.

The official original split files contain 1,934 training, 946 validation, 943
writer-dependent, and 1,797 writer-independent images. The first three are
concatenated, in that order, to form `optdigits.tra`; the writer-independent
file forms `optdigits.tes`. The generator pools every original bitmap into
$4\times4$ blocks and refuses to write the companion asset unless all 5,620
pooled rows exactly reproduce the official reduced files.

After decompressing the four `optdigits-orig.*.Z` files, regenerate with:

```sh
npm run build:ml-originals -- \
  optdigits-orig.tra optdigits-orig.cv optdigits-orig.wdep \
  optdigits-orig.windep optdigits.tra optdigits.tes
```


## All-digit assets (Chapter 4)

`optdigits-all.csv` keeps every official row, all ten labels, in the same
format and row order as the 3/8 subset:

| Split | Rows | Per label |
|---|---:|---|
| Training | 3,823 | 376, 389, 380, 389, 387, 376, 377, 387, 380, 382 |
| Test | 1,797 | 178, 182, 177, 183, 181, 182, 181, 179, 174, 180 |

Its SHA-256 is

`a14568e64aee337b66d9f570145521388e7f438248780583fd79ab993f22649b`

`optdigits-all-originals.bin` is the matching 32-by-32 companion for all 5,620
images, same header and packing as the 3/8 file, 719,380 bytes, SHA-256

`5a324ca7a966417d5f9145603cf409da1baefcae46ef6352e54078a0d6f4046e`

Regenerate both with the `--labels all` flag:

```sh
node tools/build-ml-dataset.mjs --labels all .ml-source/optdigits.tra .ml-source/optdigits.tes
node tools/build-ml-originals.mjs --labels all \
  .ml-source/optdigits-orig.tra .ml-source/optdigits-orig.cv \
  .ml-source/optdigits-orig.wdep .ml-source/optdigits-orig.windep \
  .ml-source/optdigits.tra .ml-source/optdigits.tes
```

Both builders check every per-label count against the official split and, for
the companion, re-pool all 5,620 originals and compare them with the official
reduced rows before writing. Running either without `--labels all` still
reproduces the 3/8 assets byte for byte.

## Where the raw source files live

`.ml-source/` in the repository root holds the six extracted UCI files and is
gitignored. It is the default place to point the builders. If it is missing,
re-download the archive; the recorded SHA-256 values above and in the
reproduction section will confirm the download.