A subject-independent study · Columbia Gaze · 56 people
Is it looking at you?
How many labelled photos does a model need to tell when someone looks into the camera, for a person it has never seen? We measured it for four kinds of frozen vision model, from 5 examples per class to all of them.
gaze H+0°
gaze V+0°
Columbia labelnegative
gaze-locked
5,880images at 5184 × 3456 px
56people, never shared between train and test
3,160training runs, each logged with its exact samples
1,000subject-bootstrap draws behind every interval
0numbers changed by a rerun from a clean commit
The 5° problem
Looking at the lens and looking just past it differ by a few pixels.
The iris drawn here sits where it really sits: the median measured position across the 56 people at each gaze angle, with the head facing the camera. The faint band shows how much people differ.
–eye-widths: how far the iris moves between 0° and +5°
–eye-widths: how much people already differ at 0° (interquartile range)
The signal and the person-to-person spread are about the same size, and the study's main result follows from that.
What the best model says, target by target
Share of images the geometry model (25 training examples per class, unseen people) calls "looking at the camera". The ringed centre is the only true positive.
Learning curves
Nine hand-built measurements beat a foundation model at every budget.
Experiments not yet run:
site/data/results.json is missing.Held-out people only. Each point is the mean of 5 subject folds × 5 sampling seeds; bands are 95 % subject-bootstrap intervals. Dashed: methods that use no labels from this study. ALL ≈ 225 positives and 4,480 negatives per fold.
Head pose
AUROC within each head pose with all training data; 56 positives per pose; bars are 95 % intervals.
Wrong labels
k = 100 per class. A share of each class's training labels is flipped; test labels stay clean.
Study 2 · inside the models
Where does gaze live inside a foundation model?
We read out every layer of five frozen models, from the summary token and from the patches sitting on the eyes, and asked a linear decoder to recover the gaze angle of people it had never seen.
Study 2 results are not in this build yet.
Held-out-subject R² of a ridge decoder at each layer (0 = patch embedding, 12 = last block). For each model, top row: summary token; bottom row: eye patches. Stronger colour means more decodable; values below 0 show as 0. For reference, the nine geometry measurements reach the R² shown in the key. Hover a cell for its value.
Live demo · runs in your browser
Point your camera at a face.
This is the study's best model, the geometry classifier trained on all 5,878 images, running on MediaPipe in your browser. Video and photos never leave your device. It learned from 56 people in one lab at 2 m, so treat its answer as an experiment, not a verdict.
Start the camera or choose a photo.
The face model (about 13 MB) loads on first use.
Ready.
–no face yet
Live probability over the last 10 seconds; the line marks the 0.5 threshold.
The nine measurements
| Feature | Now | Training range |
|---|
How it was done
Built so the numbers can be trusted.
Audit the data
All 5,880 images were checked. One byte-identical pair with conflicting gaze labels was found and removed before any modelling.
Split by person
Five folds by identity. Tests prove no person is ever on both sides, and every method sees the same test images.
Share every sample
For each fold, seed and budget, one set of training images is drawn and given to every method, nested from 5 to ALL.
Commit the plan first
Classifier settings, threshold, prompts, comparisons and Study 2 were committed to git before they ran. Later additions are labelled post-hoc.
Honest intervals
95 % intervals resample whole people, not images, with the same draws for every method so differences are paired.
Rerun and compare
The full study was rerun from a clean commit. All 3,160 records matched to the last digit.
What this cannot tell you
- Not eye contact. People looked at a lens or a dot in a lab, not at someone talking to them.
- One lab. 56 people, 2 m away, a chin rest, head turns only left and right.
- Unequal prior knowledge. L2CS-Net was trained with gaze labels elsewhere, so its budget undercounts.
- No fairness claim. The dataset has no demographic labels, so subgroup performance is unknown.
- Not for monitoring. These are research models. Do not use them to watch people.
Reproduce
Every number here comes from these commands.
python scripts/audit_dataset.py --config configs/experiment.yaml
python scripts/extract_features.py --config configs/experiment.yaml
python scripts/run_experiments.py --config configs/experiment.yaml
python scripts/study2_extract.py --config configs/experiment.yaml
python scripts/study2_probe.py --config configs/experiment.yaml
python scripts/generate_report_assets.py --config configs/experiment.yamlResults from run –, commit –.