My parser grabbed the first number it saw on the back of a record sleeve. On my test photos, that number was the catalog number — top corner, right where I expected it. On a real sleeve from my own shelf, the first number Vision handed me was the barcode digits.
That was the first of four bugs I fixed in a single day in the sleeve scanner in VinylCrate, my vinyl-collection app. Four different symptoms, four different fixes, one root cause: I was treating Vision’s output as a string when it’s actually a map.
Vision doesn’t give you a document. It gives you a bag of observations — strings pinned to normalized bounding boxes, in whatever order the recognizer felt like emitting them. Reading order isn’t in there. Word boundaries aren’t in there. Layout isn’t in there. You have coordinates, and everything you think of as “the text” is something you compute from those coordinates. Skip that step and your parser works on your three test photos and fails on the first sleeve that came out of a different pressing plant.
Here’s the setup, the four bugs, and — because every one of these survived my manual testing — the fixture that now pins each one down.
What Vision Actually Hands You
The modern Swift Vision API makes the request itself almost boring. RecognizeTextRequest is a value type, perform is async, and the whole thing is clean under Swift 6 strict concurrency:
import Vision
struct TextBox: Equatable, Sendable {
var string: String
var box: CGRect
}
func recognizeText(in imageData: Data) async throws -> [TextBox] {
var request = RecognizeTextRequest()
request.recognitionLevel = .accurate
request.usesLanguageCorrection = false
let observations = try await request.perform(on: imageData)
return observations.compactMap { observation in
guard let candidate = observation.topCandidates(1).first else { return nil }
return TextBox(string: candidate.string, box: observation.boundingBox.cgRect)
}
}
Two decisions in there matter. Language correction is off because — as the comment in VinylCrate’s shipping recognizer puts it — catalog codes aren’t words. 60439-1 isn’t in anyone’s dictionary, a matrix etching is exactly the kind of string a language model “helpfully” rewrites into English, and band names and track titles are worse: half of them are stylized past anything a dictionary expects. And I immediately flatten each observation into a TextBox — a string and a rect, nothing else. Every helper downstream is a pure function over [TextBox]. That’s not an aesthetic choice. It’s what makes the fixtures possible: I can pin every geometric decision with an array literal instead of a photo shoot. (Keeping the rect earns its keep elsewhere too — the dead-wax scanner uses the same geometry to exclude the label region from live scans.)
What comes back from perform is where the trouble starts. An array of observations that looks ordered, reads ordered in the debugger for simple images, and is not ordered.
Bug 1: The Array Is Not the Sleeve
The symptom. A back cover with the classic two-column tracklist — titles down the left, runtimes down the right — came out as BATTERY, MASTER OF PUPPETS, 5:10, 8:38. My parser, walking the array looking for the first duration “after” each title, gave both tracks the same runtime — and decided an eight-and-a-half-minute track ran 5:10.
The wrong assumption. That the observation array is in reading order. It roughly tracks the image top-to-bottom for simple layouts, which is exactly the kind of roughly that survives every demo and dies in production. The moment a sleeve has columns, offset side-one/side-two blocks, or a title set on a diagonal, the recognizer interleaves regions freely. Nothing in the API contract says otherwise. I had just never read the contract.
The fix. Reading order is a spatial property, so compute it spatially. Cluster observations into lines by vertical midpoint — with a tolerance, because two boxes on the same printed line never share an exact midY — then sort lines top-to-bottom and members left-to-right:
func clusterLines(_ boxes: [TextBox]) -> [[TextBox]] {
guard !boxes.isEmpty else { return [] }
let sorted = boxes.sorted { $0.box.midY < $1.box.midY }
let medianHeight = sorted.map(\.box.height).sorted()[sorted.count / 2]
let tolerance = medianHeight * 0.6
var lines: [[TextBox]] = []
var current = [sorted[0]]
for box in sorted.dropFirst() {
let lineMidY = current.map(\.box.midY).reduce(0, +) / CGFloat(current.count)
if abs(box.box.midY - lineMidY) <= tolerance {
current.append(box)
} else {
lines.append(current)
current = [box]
}
}
lines.append(current)
return lines.map { $0.sorted { $0.box.minX < $1.box.minX } }
}
The tolerance is relative to the median box height, not a magic constant in normalized units, so it holds up whether the sleeve fills the frame or sits in a corner of the photo.
The fixture. Four boxes, deliberately fed in the interleaved order Vision actually produced. No image required — the geometry is the fixture:
@Test func twoColumnTracklistReadsRowByRow() {
let boxes = [
TextBox(string: "BATTERY", box: CGRect(x: 0.05, y: 0.10, width: 0.20, height: 0.04)),
TextBox(string: "MASTER OF PUPPETS", box: CGRect(x: 0.05, y: 0.18, width: 0.40, height: 0.04)),
TextBox(string: "5:10", box: CGRect(x: 0.55, y: 0.10, width: 0.08, height: 0.04)),
TextBox(string: "8:38", box: CGRect(x: 0.55, y: 0.18, width: 0.08, height: 0.04)),
]
let lines = clusterLines(boxes).map(joinLine)
#expect(lines == ["BATTERY 5:10", "MASTER OF PUPPETS 8:38"])
}
If anyone — including me, six months from now — “simplifies” the clustering back into an array walk, this test fails with a message that explains the whole bug.
Bug 2: Y Grows Up
The symptom. The first version of clusterLines shipped with a different sort, and the output read like the sleeve had been fed through the scanner bottom-up. Barcode and ℗ line first, album title last. Perfectly ordered. Perfectly backwards.
The wrong assumption. That origin.y means what UIKit has trained me to think it means. Vision’s normalized coordinate space puts the origin at the bottom-left, y increasing upward — image-processing convention, not screen convention. Sorting by minY ascending walks the sleeve from the bottom. Worse, any box math you do in the wrong space is quietly broken: “is this box above that one” flips meaning, and vertical-overlap checks compare the wrong edges.
The fix. Convert once, at the boundary, and let everything downstream live in top-left space. If you’re projecting into image pixels anyway, the API will do it for you — boundingBox.toImageCoordinates(imageSize, origin: .upperLeft). For normalized-space work, the flip is one line:
func topLeftRect(_ visionRect: CGRect) -> CGRect {
CGRect(x: visionRect.minX,
y: 1 - visionRect.maxY,
width: visionRect.width,
height: visionRect.height)
}
recognizeText now applies this before anything else sees a rect. One conversion, one place. The alternative — sprinkling 1 - y through the codebase wherever things look upside down — is how you end up with code that flips twice and looks correct in exactly one screen orientation.
The fixture. Two boxes in raw Vision space: the album title near the top of the sleeve (high y, in Vision’s terms) and the ℗ line near the bottom. The test pins the relationship, not the arithmetic:
@Test func visionSpaceFlipsToReadingOrder() {
let albumTitle = CGRect(x: 0.05, y: 0.85, width: 0.40, height: 0.05)
let copyrightLine = CGRect(x: 0.05, y: 0.05, width: 0.30, height: 0.04)
#expect(topLeftRect(albumTitle).midY < topLeftRect(copyrightLine).midY)
}
It looks almost too small to bother with. It has already caught one refactor where a well-meaning cleanup removed the “redundant” flip.
Bug 3: Word Boundaries Are Load-Bearing
The symptom. MASTEROFPUPPETS. Also, on a different sleeve, METALLI CA — the same code, failing in the opposite direction.
The wrong assumption. That joining strings is a string problem. My first pass used joined() — words fused. The obvious “fix” was joined(separator: " ") — and now words that Vision had split mid-word across two observations grew a space in the middle. There is no string-level answer here, because whether two observations deserve a space between them isn’t a property of the strings. It’s a property of the gap between their boxes.
The fix. Measure the gap and compare it to the local character width. If the horizontal distance between two boxes is a meaningful fraction of an average character in those boxes, it’s a word boundary. If it’s a sliver, it’s the same word:
func joinLine(_ line: [TextBox]) -> String {
guard var result = line.first?.string else { return "" }
for (previous, next) in zip(line, line.dropFirst()) {
let gap = next.box.minX - previous.box.maxX
let averageCharWidth = (previous.box.width + next.box.width)
/ CGFloat(previous.string.count + next.string.count)
result += gap > averageCharWidth * 0.35 ? " " + next.string : next.string
}
return result
}
The 0.35 came from measuring real sleeves, and I’ll be honest: it’s a heuristic, and heuristics over photographs of forty-year-old ring-worn cardboard have losing days. But it’s a heuristic about geometry, which means it degrades gracefully — where joined(separator: " ") was a coin flip dressed up as a decision.
The fixture. Both directions, in one test, because this bug fails both ways and a fix for one direction can silently reintroduce the other:
@Test func wordBoundariesComeFromGeometryNotLuck() {
let title = [
TextBox(string: "RIDE", box: CGRect(x: 0.05, y: 0.40, width: 0.10, height: 0.04)),
TextBox(string: "THE", box: CGRect(x: 0.17, y: 0.40, width: 0.08, height: 0.04)),
TextBox(string: "LIGHTNING", box: CGRect(x: 0.27, y: 0.40, width: 0.22, height: 0.04)),
]
#expect(joinLine(title) == "RIDE THE LIGHTNING")
let splitWord = [
TextBox(string: "METALLI", box: CGRect(x: 0.050, y: 0.10, width: 0.21, height: 0.05)),
TextBox(string: "CA", box: CGRect(x: 0.266, y: 0.10, width: 0.06, height: 0.05)),
]
#expect(joinLine(splitWord) == "METALLICA")
}
The rects in splitWord are lifted straight from the failing sleeve’s debug output. That’s the move worth stealing: when a real image breaks you, don’t just fix the code — harvest the coordinates into a fixture before you lose the photo.
Bug 4: Proximity Beats Order
The symptom. One track’s runtime came back as 60439-1 — the catalog number. The parser found the track title, then took “the next observation” as its duration. In array order, the next observation was whatever the recognizer emitted next — which, per bug 1, could be anything on the sleeve, including the catalog block.
The wrong assumption. The subtlest one, because it felt fixed. I had line clustering. I had geometric joining. And then, for title–duration association, I reached for array order one more time — firstIndex(of:) plus one. The title and its runtime sit on the same printed line, but on a two-column tracklist they can arrive dozens of positions apart in the array, with half the sleeve between them.
The fix. The value for a label is the nearest observation to its right on the same baseline — a geometric query, not an index:
func verticalOverlap(_ a: CGRect, _ b: CGRect) -> CGFloat {
let overlap = min(a.maxY, b.maxY) - max(a.minY, b.minY)
return overlap / min(a.height, b.height)
}
func value(for label: String, in boxes: [TextBox]) -> TextBox? {
guard let anchor = boxes.first(where: {
$0.string.localizedCaseInsensitiveContains(label)
}) else { return nil }
return boxes
.filter {
$0.box.minX > anchor.box.maxX
&& verticalOverlap($0.box, anchor.box) > 0.5
}
.min {
($0.box.minX - anchor.box.maxX) < ($1.box.minX - anchor.box.maxX)
}
}
Requiring 50% vertical overlap — rather than a midY distance — handles boxes of different heights sharing a baseline, which happens constantly when a track title set in one weight sits next to a runtime set in another. And min by horizontal distance means a barcode two columns over never wins against the duration printed on the same line.
The fixture. The array order is the trap, so the fixture bakes the trap in — the catalog number sits between the title and its runtime, exactly where the old code would grab it:
@Test func nearestOnBaselineBeatsNextInArray() {
let boxes = [
TextBox(string: "ORION", box: CGRect(x: 0.05, y: 0.60, width: 0.10, height: 0.04)),
TextBox(string: "60439-1", box: CGRect(x: 0.30, y: 0.72, width: 0.14, height: 0.04)),
TextBox(string: "8:12", box: CGRect(x: 0.55, y: 0.60, width: 0.08, height: 0.04)),
]
#expect(value(for: "ORION", in: boxes)?.string == "8:12")
}
The catalog number even passes the “is it to the right of the anchor” check. It dies on the vertical-overlap filter — it’s on a different line, and no amount of array adjacency changes that.
One Fixture That Is a Photo
Pure-geometry fixtures carry most of the weight, but they all share one blind spot: they trust my mental model of what Vision emits. So the suite ends with a fixture that is an image — a sleeve pulled off my own shelf, back cover photographed flat, checked into the test bundle, and run through the real request:
I wrote the obvious assertions first: the joined lines should contain MASTER OF PUPPETS 8:38, and value(for: "ORION") should hand back 8:12, because that’s what’s printed on the sleeve.
The photo failed both immediately — and the failure is the most instructive thing in this post. The runtimes on this pressing are set in type small enough that Vision doesn’t emit them as observations at all. It fuses a garbled fragment into the title’s own box: ORION •1, one observation, one rect. There is no duration box anywhere in the output. value(for: "ORION") isn’t wrong — it’s nil, because the thing I was querying for doesn’t exist in the map. Tiny type claimed other casualties too: CUFF BURTON.
So the test pins what the pipeline actually guarantees on this photograph — rows pair across the column gutter, reading order runs top-down, the nearest-on-baseline query finds the true row partner, and the missing duration stays honestly missing:
@Test func realSleevePhotoSurvivesTheWholePipeline() async throws {
let url = try #require(
Bundle.module.url(forResource: "sleeve-back-two-column", withExtension: "heic")
)
let boxes = try await recognizeText(in: Data(contentsOf: url))
let lines = clusterLines(boxes).map(joinLine)
let puppetsRow = try #require(lines.first { $0.contains("MASTER OF PUPPETS") })
#expect(puppetsRow.contains("LEPER MESSIAH"))
let batteryIndex = try #require(lines.firstIndex { $0.contains("BATTERY") })
let creditsIndex = try #require(lines.firstIndex { $0.contains("PRODUCED BY") })
#expect(batteryIndex < creditsIndex)
let rowPartner = value(for: "THE THING THAT SHOULD NOT BE", in: boxes)
#expect(rowPartner?.string.hasPrefix("ORION") == true)
#expect(value(for: "ORION", in: boxes) == nil)
}
That last #expect is the one I’d defend in review. It documents a real property of real input — on this sleeve, at this type size, durations are not recoverable as separate fields — and it will fail loudly the day a Vision update starts splitting them, which is exactly when I’d want to revisit the parser.
This test is slower than the others and slightly at the mercy of the recognizer’s mood across OS releases. It’s also the only one that would have caught all four bugs at once, because it exercises the actual contract: observations in whatever order, in Vision’s coordinate space, split — or fused — wherever the recognizer chose. The array-literal fixtures explain which assumption broke. The photo proves the assumptions cover reality, and when the two disagree, the photo wins the argument.
The Map Is the Territory You Get
Four bugs, and every one of them was the same bug wearing a different jacket: I kept treating a spatial data structure as a sequential one. The array order lied. The coordinate origin lied. The whitespace never existed. The “next” element was a stranger from another column.
OCR output is a map, not a string. Reading order is something you compute from y. Word boundaries are something you compute from gaps. Field association is something you compute from proximity — because on a printed page, proximity beats order every single time.
And every geometric assumption you make deserves a fixture, because these bugs have a signature failure mode: they work on your three test photos. The sleeve that breaks you will come from a pressing plant you’ve never seen, with a layout you didn’t imagine, and the only question is whether the coordinates it exposes end up in your test suite or in your support inbox.
Mine go in the test suite now. Harvest the rects, pin the behavior, keep the photo.