Frame TV art project · side experiment
16:9 Crop Bake-off
I built a public-domain art collection, north of 60 pieces, for my Samsung Frame TV, which displays
everything at 16:9. Most of the source paintings and prints aren't natively that shape, so each one
needed cropping. I had Claude Code (running on Sonnet 4.6) do that: look at each image and judge a
sensible crop window. Then I got curious whether an automated saliency or smart-cropping tool could
do the same job. I picked 10 representative images (faces and figures, abstract geometry, wide
panoramic scrolls, ordinary landscapes) and ran each one through three different automatic cropping
approaches, checked against Claude's crop.
Takeaways
- When there's an unambiguous focal point, a face, a wave, a shipwreck, all three tools land on almost the same crop Claude landed on. The Scream, Hokusai, and Aivazovsky all came out nearly identical across the board.
- The Mondrian is where they fall apart. smartcrop.py and Apple Vision both fixated on the long black double-lines and cropped the painting's only spot of color, a small orange block, right out of frame. Only Google's Crop Hints kept it.
- None of the three understand text. Every automated tool pulled in at least a sliver of the calligraphy panel on the Korean handscroll, because dense script reads as high-contrast and "salient" even though any human curator would ignore it on sight.
- smartcrop.py costs nothing, no network call, no model download, and mostly kept pace with the other two. But it lost badly twice: it fixated on tree branches and missed the sea entirely in the Friedrich, and it cropped the shipwrecked sailors almost out of frame in the Aivazovsky.
- If I had to pick one tool, it's Google's Crop Hints. smartcrop.py is a decent free fallback for anything with an obvious subject.