Imagine asking your phone:
“Call ”
“Turn on the flashlight”
…and it obeys instantly — without sending your voice to a server across the world.
No internet.
No cloud GPU.
No latency.
Just pure, on-device intelligence.
That’s exactly what I built using Google’s FunctionGemma and a modified version of the Mobile Edge Gallery app. In this article, I’ll show how a regular Android phone can become an autonomous, offline AI agent using Edge AI.
The Problem: AI Is Usually “Heavy”
Most AI assistants today live in the cloud.
When you ask them to do something:
- Your data leaves the device
- It’s processed on massive server farms
- The response comes back
This introduces three fundamental problems:
- Latency — Cloud round trips are slow
- Privacy — Your voice and intent leave your device
- Dependency — No internet = no intelligence
That’s not intelligence — that’s outsourcing thinking.
The Solution: Tiny, Mighty, and Fully Local
Instead of moving data to the brain, I moved the brain to the phone.
Here’s the exact recipe.
1. The Brain: FunctionGemma 270M (Fine-Tuned by Me)
I started with FunctionGemma, a specialized variant of Google’s Gemma models designed not just to talk, but to call functions.
Why FunctionGemma?
Because I didn’t want poetic responses — I wanted actions.
When a user says:
“I need to take a picture”
The model shouldn’t explain photography — it should output:
open_camera()
My Fine-Tuning Process
- I fine-tuned the 270M parameter version (yes, tiny)
- Training data focused entirely on Mobile Actions
- Used Google’s official Colab notebook for function tuning
👉 Fine-tuning notebook
The Result
A lightweight LLM that understands intent → action, not intent → text.
📦 Download the fine-tuned model
👉 FunctionGemma 270M Mobile Actions (LiteRT)
2. The Translator: LiteRT (TensorFlow Lite Runtime)
Raw models are too slow and too heavy for mobile devices.
So I converted the fine-tuned model into LiteRT (.litertlm) format.
Why LiteRT?
- Optimized for mobile CPUs
- No GPU or NPU required
- Runs smoothly on most modern Android phones
- No overheating, no battery drain panic
This makes true offline AI practical, not theoretical.
3. The Body: Modified Mobile Edge Gallery App
Intelligence without action is useless.
So I took Google’s Mobile Edge Gallery app and slightly modified it to support custom mobile actions.
Accessibility Service (The Secret Sauce)
I added a custom Android Accessibility Service — a privileged background service that can:
- Observe UI state
- Simulate gestures
- Trigger system APIs
The Execution Loop
Here’s what happens in real time:
- User taps the mic and says
“Turn on the flashlight” - Edge AI processes the command locally
- Model outputs
turnOnFlashlight()
- App parses the function call
- Accessibility Service triggers the Torch API
- Flashlight turns ON
All of this happens in milliseconds — completely offline.
How to Try It Yourself
Want to experience real Edge AI?
Step 1: Download the Model
👉 FunctionGemma 270M LiteRT Model
Step 2: Install the Modified App
👉 Download Modified Mobile Edge Gallery APK
Step 3: Setup
- Open the app and load the downloaded model
- Go to Settings → Accessibility
- Enable Mobile Actions Service
- Grant required permissions:
- Overlay
- Read Contacts
- Phone access
Step 4: Magic ✨
Tap the floating red mic and command your phone.
Why This Matters (Beyond a Demo)
This isn’t just a fun experiment — it’s a preview of the future.
Privacy-First Computing
Your voice, intent, and actions never leave your device.
Zero-Dependency Intelligence
Works:
- In tunnels
- On flights
- In remote locations
- Without SIM or Wi-Fi
♿Accessibility Superpowers
Voice-controlled, intent-aware UI can radically improve device access for users with motor impairments — far beyond rigid command systems.
Final Thoughts
Edge AI isn’t coming.
It’s already here.
It’s fast.
It’s private.
And it fits in your pocket.
The future won’t be cloud-only — it’ll be local, intelligent, and autonomous.
And this is just the beginning.
🚀 How I Built an Offline AI Assistant That Controls Android Phone. was originally published in Coinmonks on Medium, where people are continuing the conversation by highlighting and responding to this story.