Implementing Google ML Kit with Antigravity: What the Docs Don't Tell You

When you spend years building wallpaper and photo apps — I've been doing independent app development since 2014, and those apps have been downloaded over 50 million times combined — user requests for "read the text in this image" start showing up more and more. OCR (optical character recognition) sounds straightforward until you try to wire it into a SwiftUI camera view using Google ML Kit and hit something the official docs quietly skip over.

Working through the implementation with Antigravity cut what would have been a one-to-two day integration down to half a day. Here's what I found, including the one place things got stuck.

How ML Kit Compares to Core ML and TFLite

There are several ways to run AI on-device in a mobile app. If you're building Apple-only, Core ML gives you the best performance. But if you need iOS and Android in one codebase built by a single developer, you need something else.

Google ML Kit is an on-device AI SDK from Google, tied to the Firebase platform. Under the hood it uses TensorFlow Lite (TFLite), but it wraps everything in a high-level API so you never touch the raw TFLite model layer. The key selling point: popular features like text recognition, face detection, barcode scanning, and pose estimation come pre-packaged with their models already included.

A quick comparison:

• Core ML: Apple-only. Uses .mlmodel files. Best performance on iOS/macOS, but no Android
• TFLite: Google's cross-platform ML. Flexible for custom models, but lower-level setup
• ML Kit: High-level API on top of TFLite. Easiest to start with, supports both iOS and Android

For a solo developer shipping to both platforms, ML Kit is a realistic starting point. Once the setup is done, text recognition takes a few dozen lines of code.

Adding ML Kit in Antigravity (via Swift Package Manager)

ML Kit supports both CocoaPods and Swift Package Manager (SPM). With Xcode 15 or later, SPM is the stable choice.

I told Antigravity: "I want to add ML Kit text recognition via SPM and connect it to a SwiftUI camera view." It generated the dependency declaration, the camera permission entries for Info.plist, and the AVFoundation bridge code in one pass.

// Add to Package.swift dependencies
.package(
    url: "https://github.com/google/mlkit-ios-official",
    from: "6.0.0"
)

Here's the catch the docs don't mention: ML Kit binaries ship as XCFrameworks, and there's a known SPM linking bug in Xcode 15.0.1 and earlier. Antigravity identified the root cause from the error message and suggested either upgrading Xcode to 15.2+ or switching to CocoaPods. I upgraded Xcode and that resolved it. If you're running CI, check your Xcode version before anything else.

Text Recognition Implementation (iOS)

ML Kit's text recognizer uses the TextRecognizer class. Antigravity generated the full wiring between SwiftUI view, AVCaptureSession, and the recognizer — but thread management is where things get subtle.

import MLKitTextRecognition
import MLKitVision
import AVFoundation
 
// Detect text from a live camera frame
func recognizeText(from sampleBuffer: CMSampleBuffer) {
    let visionImage = VisionImage(buffer: sampleBuffer)
    visionImage.orientation = imageOrientation()  // account for device rotation
 
    let recognizer = TextRecognizer.textRecognizer()
    recognizer.process(visionImage) { [weak self] result, error in
        guard error == nil, let result = result else { return }
        let detectedText = result.text
        DispatchQueue.main.async {
            // UI updates must happen on the main thread
            self?.detectedString = detectedText
        }
    }
}
 
// Map device orientation to UIImage.Orientation
func imageOrientation() -> UIImage.Orientation {
    switch UIDevice.current.orientation {
    case .landscapeLeft:  return .up
    case .landscapeRight: return .down
    case .portraitUpsideDown: return .left
    default: return .right
    }
}

The visionImage.orientation mapping is the thing that trips people up. The image from a CMSampleBuffer rotates with device orientation, so you need the conversion above. Antigravity included this automatically, but note that it can't be verified on the simulator — test on a real device.

Face Detection: What the Docs Leave Out

When I tried face detection, I noticed that real-time processing accuracy was noticeably lower than snapshot mode. The official documentation has nothing about accuracy differences between real-time and still-image processing.

Dropping the frame rate (30fps → 15fps) and switching performanceMode to .accurate helped, at the cost of added latency:

let options = FaceDetectorOptions()
options.performanceMode = .accurate  // default is .fast
options.landmarkMode = .all          // eyes, nose, mouth landmarks
options.classificationMode = .all    // smile, open/closed eyes
 
let faceDetector = FaceDetector.faceDetector(options: options)

If you need real-time face detection, plan the accuracy vs. latency trade-off early. For this particular app, I ended up switching to snapshot mode for face detection rather than processing every frame.

Where Antigravity Helped — and Where It Didn't

Where it helped:

• Diagnosing the SPM error and identifying the Xcode version as the cause (instant)
• Generating the full AVFoundation + ML Kit connection boilerplate
• Properly separating main thread UI updates from background processing

Where it didn't:

• Diagnosing bugs that only reproduce on real hardware (proposals assumed simulator behavior)
• Reflecting post-2025 ML Kit spec changes (some suggestions were based on older API versions)

For the second point: telling Antigravity "please check the latest official documentation before answering" shifted its responses toward more careful, qualified proposals. The tool's behavior changes based on how you direct it.

Practical Notes for Indie Developers

Something I've noticed across over a decade of solo app development: the availability of pre-built models has quietly changed what's feasible alone. When you don't have time to train custom models, a pre-built SDK with good-enough accuracy is a practical choice.

That said, watch the Firebase dependency depth. ML Kit's on-device features are free, but combining them with Cloud APIs introduces billing. In the early stages of an indie app, start with purely on-device features before adding cloud calls.

The workflow that worked best: run the Google ML Kit samples repository locally first, then ask Antigravity to help port the relevant parts to your specific project structure. It reads the sample code and proposes how to adapt it — which turns out to be a strong use case for AI-assisted development.

For related reading, on-device AI with Core ML and TFLite in Antigravity covers the lower-level options, and Xcode × Antigravity iOS development guide goes deeper on the Xcode setup side.