Updated
Family Photo Archive
A box of old family prints, and how far off-the-shelf AI can go at restoring and understanding them.
family prints and Polaroids, scanned at 600 DPI and put through an AI restoration pipeline in three days, for $21.73.
- Scan
- Crop
- Restore
- Enhance
- Describe
- Gallery
The attempt
A flatbed scanner, a box of prints from several decades, some of them Polaroids, some with handwriting on the back. The plan was to preserve them, make them browsable by era, mood and the people in them, and find out how much of the restoration and cataloguing work current AI models can do on their own.
The pipeline is non-destructive. Every stage writes its own layer, so any photo can be compared before and after any step, and every automated decision can be overridden by hand. The result was a gallery in English, Russian and Hebrew; this page is the story of how it was made.
The collection
A first pass by GPT-4o sorted every scan before anything touched it: black-and-white or sepia, faded, torn, cracked, which colour cast it had.
- 151 black and white
- 19 sepia
- 62 colour
The pipeline
-
Scan
Flatbed at 600 DPI on an A4 bed, several prints per scan. Backs are scanned too, next to their fronts.
-
Triage
GPT-4o looks at a 1024 px thumbnail and returns a processing plan: Polaroid or not, damage, colour cast, how hard to correct. At 512 px it missed the cracks.
- GPT-4o
- detail: high
-
Crop
Each print is found on the white scanner bed with a threshold cascade (235 → 220 → 200 → 180) and contour detection. Deskew only runs above 2°, because rotating costs about 26% sharpness. A local vision model checks every crop, and a hand-built tool fixes the rest.
- OpenCV
- Qwen2.5-VL 7B on Ollama
- manual crop tool
-
Colour cast
Casts are measured and neutralized on the a* and b* channels in LAB space. On test photos a red cast dropped from 43 to 6, a yellow one from 22 to 7.
- LAB colour space
- NumPy
-
Colorize
Black-and-white and sepia prints are colorized in the cloud: FLUX Kontext first, DDColor as the fallback.
- Replicate
- FLUX Kontext
- DDColor (large)
-
Enhance
FLUX Kontext's restore model runs on every crop as a parallel layer, fixing scratches and dust in one pass, about a cent a photo.
- FLUX Kontext restore-image
-
Committee
Three models describe, date and tag every photo in parallel. Era, mood, setting and people count go by majority vote, GPT-4o breaks ties, and every tag keeps the share of models that agreed on it.
- GPT-4o
- Gemini 2.5 Pro
- Claude Opus 4.6
-
Faces
Face embeddings are clustered so the same person can be found across decades. Naming the clusters was left for later.
- InsightFace
- DBSCAN
- ONNX Runtime
-
Gallery
A masonry gallery with tag search, era filters and before/after layers, in three languages with right-to-left Hebrew. Metadata lives in git, images on S3.
- React 19
- Vite
- AWS S3
- GitHub Pages
Where the $21.73 went
Storage for about 1.5 GB of layers on S3 costs three cents a month.
Inside the committee
The committee is the brain of the project. Instead of trusting one model, every photo is shown to three vision models at once, each answering the same questionnaire on its own. Their answers are then merged by fixed rules, so the result is reproducible and every disagreement stays visible.
The questionnaire
Each model gets the same prompt and must answer in JSON: a one-line description, how many people, the decade judging by clothes, colours and style, indoor or outdoor, a guess at the place, the mood, notable objects, searchable tags, dominant colours, the photo's condition, and whether it is a Polaroid, shows text or shows faces. The three calls run in parallel threads; if one model fails, the other two still count.
How the answers are merged
| Field | Rule |
|---|---|
| Description | The longest of the three, since it usually carries the most detail. |
| Era, people count, setting, mood | Majority vote. When all three disagree, GPT-4o's answer wins. |
| Tags | The union of all three lists, lower-cased and de-duplicated. Each tag records which models proposed it and a confidence equal to the share that did. |
| Objects | The union of all three lists. |
| Agreement score | The average confidence of the photo's tags: 1.0 if every model proposed every tag, 0.33 if they shared nothing. |
| Raw answers | Kept in the photo's metadata next to the merged result, and any merged field can be overridden by hand. |
- 1.0 · all three models
- 0.67 · two models
- 0.33 · one model
It rarely agreed
Across the collection the average agreement was 0.44, with photos ranging from 0.33 to 0.58. That is why the gallery showed a confidence on every tag instead of pretending there was one right answer.
- 6,575 tags proposed
- 1,736 distinct tags
- 329 unanimous
More jobs than tagging
The design gave the committee a vote on every judgement call in the pipeline, not just tags. Some of those roles ran, some were built and then set aside, and some stayed on paper.
| Role | What it decides |
|---|---|
| Tagging ran | Description, era, mood, setting, objects and tags for all 232 photos. |
| Triage ran | Photo type, damage and a processing plan. Adapters exist for every model, but the final run used GPT-4o alone. |
| Quality judge built | Five colour variants (gentle, moderate, strong, white balance only, CLAHE only) shown side by side, each model voting for the most natural one. Set aside once plain cast removal proved better. |
| Crack assessment built | Find cracks and judge how well they were inpainted. |
| Back-text reading built | Two readers of the handwriting on the backs, keeping the most confident reading. Never run on the collection. |
| FLUX vs pipeline designed | Compare the FLUX result with the step-by-step result and choose the default view. |
| Restoration and order designed | Whether a photo needs face restoration, and whether to denoise before or after colour correction. |
The committee started with four members. A local Qwen2.5-VL model was dropped from tagging and kept only for checking crops.
The original design
Before any code, the project was planned as a sixteen-step adaptive pipeline in which the committee could switch steps on or off for each photo. Building it was mostly a process of finding out which steps earned their place.
- ranused on the collection
- builtwritten, then set aside
- designedplanned only
Scanner profile fix built
Average the white scanner bed around all 232 scans into a flat-field map, then divide each scan by it to remove vignetting and colour cast.Committee triage ran
Classify each scan and decide which later steps to run.Smart crop ran
Find the print on the scanner bed. The final version uses a threshold cascade instead of edge detection.Rotation fix ran
Straighten scanning skew, but only above 2° to protect sharpness.FLUX enhancement ran
A one-pass AI restoration running beside the step-by-step pipeline.Crack healing built
Mask cracks and scratches and inpaint them, with OpenCV first and LaMa later.Colour variants and judge built
Five correction strategies, with the committee voting on a side-by-side grid.Colour correction ran
Shipped as cast removal in LAB space and nothing else.AI restoration built
GFPGAN for faces and Real-ESRGAN for denoising at 1×, on the Mac's GPU.Detail enhancement built
Local contrast and sharpening on the luminance channel. Tried early, then dropped with the rest of the pre-processing.Colorization ran
Only for black-and-white and sepia prints, detected by their low saturation.Borders designed
Three finishing borders, with the committee choosing one per photo.FLUX vs pipeline vote designed
Pick the better of the two results as each photo's default.Face clustering ran
512-dimensional face embeddings grouped with DBSCAN: name a face once and it is found everywhere.Photo tagging ran
The committee's questionnaire and merge, above.OCR and gallery built
Apple Vision reading Russian handwriting on the backs, feeding the gallery. The gallery shipped; the OCR never ran.
Polaroids
A Polaroid's thick white frame throws off both colour correction and the AI's reading of the picture, so the 27 Polaroids got their own path. Crop rules know the frame's proportions: an 89 × 108 mm print whose bottom border is almost five times deeper than the top.
- Find the outer frame
- Find the picture inside it
- Process only the picture
- Put it back in the frame
- Even out the border
- Check the crop by eye or model
Nothing is overwritten
Every stage writes its own folder, so one photo exists as a stack of versions. The gallery could show any layer and slide between them, and a bad result at one stage never destroyed the one before it.
cropped/The print alone: no scanner bed, no border, straightenedcorrected/Colour cast removedcolorized/Black-and-white and sepia prints in colourflux-enhanced/The FLUX Kontext restorationthumbnails/800 px versions for the web and the committeemetadata/One JSON file per photo, every model's answer included, plus an index
What I learned
Worked
- Plain computer vision beat the AI models at cropping. GPT-4o's crops were unusable; a threshold cascade got 98%.
- Less pre-processing before the AI. Auto-levels, CLAHE and white balance were all dropped, leaving only cast removal.
- FLUX Kontext colorizes and removes scratches in one pass, and beat DDColor, DeOldify and manual LAB correction.
- Keeping every stage as its own layer made each experiment cheap to compare and undo.
Didn't
- FLUX faces were less trustworthy than the rest of the image.
- DeOldify had broken dependencies, DDColor wouldn't install locally, and InstantIR didn't run on Apple Silicon.
- Git LFS was the wrong home for 1.5 GB of layers, so images moved to S3.
- Left unfinished: face-preserving restoration with DiffBIR, naming the face clusters, and reading the handwriting on the backs. The OCR stage was built but never run.
Built with
Started