Google AI Edge Eloquent: Running Google's AI Models Natively on iOS

Running Google's AI models on an iPhone without a network connection would have sounded optimistic a couple of years ago. With Google AI Edge Eloquent, it's a practical implementation choice.

Eloquent is the iOS and Apple Silicon component of Google's AI Edge platform — the counterpart to LiteRT (formerly TensorFlow Lite) on Android. It lets you load and run .task format Google models locally on Apple hardware, with streaming token generation and straightforward Swift integration.

The Google AI Edge Platform

Before diving into Eloquent specifically, it helps to understand where it fits:

• LiteRT (formerly TensorFlow Lite): On-device inference for Android and IoT devices
• Eloquent: On-device inference for iOS, macOS, and Apple Silicon
• MediaPipe: Cross-platform AI pipeline framework (works above both)
• Android AI Core: Gemma runtime integration for Android devices

Eloquent isn't competing with Apple's Core ML — it's a layer that lets you run Google's model formats efficiently on Apple hardware. The use case is specific: you want to ship Gemma or another Google model in an iOS app without routing data through a server.

Why Choose Eloquent Over Core ML or ONNX Runtime?

iOS developers already have options for on-device inference:

Core ML runs Apple-format models with excellent Metal optimization and full access to Apple Neural Engine. It's the right choice when you're using Apple-trained models or can convert to .mlpackage format.

ONNX Runtime for iOS offers model portability across frameworks, but the integration story with Google's model ecosystem requires extra conversion steps.

Eloquent is the most direct path when you're working with Google's .task format models — Gemma in particular. No conversion required, and the SDK handles Apple Silicon acceleration internally.

Implementation with Antigravity

Give Antigravity this prompt to generate the foundational inference code:

"Create a SwiftUI implementation using Google AI Edge Eloquent to run Gemma 
locally on iOS. Include model loading, streaming text generation, and proper 
error handling."

The generated pattern looks like this:

import GoogleAIEdge
 
class GemmaLocalInference: ObservableObject {
    private var model: EloquentTextModel?
    @Published var isLoading = false
    @Published var output = ""
    
    func loadModel() async throws {
        guard let modelPath = Bundle.main.path(
            forResource: "gemma-2b-it-cpu",
            ofType: "task"
        ) else {
            throw EloquentError.modelNotFound
        }
        
        let options = EloquentTextModel.Options()
        options.maxTokens = 512
        options.temperature = 0.7
        
        model = try await EloquentTextModel.load(
            modelPath: modelPath,
            options: options
        )
    }
    
    func generate(prompt: String) async {
        guard let model = model else { return }
        isLoading = true
        output = ""
        
        do {
            for try await token in model.generateStream(prompt: prompt) {
                await MainActor.run { output += token }
            }
        } catch {
            await MainActor.run {
                output = "Error: \(error.localizedDescription)"
            }
        }
        
        await MainActor.run { isLoading = false }
    }
}

Model Sizes and App Distribution

The .task format Gemma models are large. Planning your distribution strategy early saves a painful redesign later:

Bundling 800 MB into your main app binary will cause App Store review friction and a poor first-launch experience. The recommended approach is On-Demand Resources (ODR) — deliver the model asset after the app installs, on first launch.

// On-Demand Resources pattern for model delivery
let request = NSBundleResourceRequest(tags: ["gemma-model"])
try await request.beginAccessingResources()
// Model is now accessible — load with Eloquent

Ask Antigravity to implement the full ODR flow including progress indication, retry logic on failed downloads, and fallback behavior. It generates a robust implementation in one pass.

Practical Use Cases

Privacy-first text processing: Summarization, translation, or analysis of journals, medical notes, or personal documents that users don't want leaving the device. "Runs completely offline, no data sent to servers" is a real differentiator in privacy-conscious segments.

Offline language learning: An AI conversation tutor that works in airplane mode or the subway. Cloud-dependent language apps have a gap here that on-device AI fills cleanly.

Low-latency real-time tasks: Live captioning, real-time translation, or voice command processing where network round-trips introduce unacceptable lag.

Current Limitations

Eloquent is in Developer Preview as of 2026. Know the constraints before building:

• Limited model support: Gemma 2B and 7B variants currently; Gemma 4 support is on the roadmap
• Metal optimization is behind Core ML: Core ML has deeper access to Apple Neural Engine and Metal Performance Shaders — if raw throughput on Apple hardware is the priority and you can use Apple models, Core ML wins
• Japanese language quality: Gemma's Japanese capability trails Claude and Gemini; test your target language thoroughly before committing

For the specific case of running Google models on iOS without a server, Eloquent is the right tool. For maximum Apple Silicon performance with Apple models, Core ML remains the better choice.

Getting Started

The Google AI Edge documentation has an Eloquent quickstart. Start by adding the SDK to an Xcode project and running inference with the sample model — verifying the end-to-end pipeline before committing to the model delivery architecture.

Antigravity accelerates the "get something running" phase considerably. A prompt describing your use case — the model, the task type, the UI interaction pattern — produces a working scaffold you can iterate from rather than starting blank.