Exploring

Computer-use for Android

A native Android agent that can understand a requested task, use explicit phone capabilities, observe the result, and continue safely.

Next experiment: Create a native Android app with one verified capability: launch another app from a user command.

  • Android
  • AI agents
  • Automation
  • Product systems

The product direction

Computer-use for Android is an experiment in building an agentic control layer over a phone. The long-term loop is:

Understand -> Decide -> Act -> Observe -> Verify -> Continue

The system should eventually accept a request such as “Open my files, find the PDF I downloaded today, move it into my Physics folder, and tell me when it is done.” It would then turn that request into a bounded plan, use explicit Android capabilities, inspect what happened, and only continue when the result is verified.

The important constraint is that the agent should not receive unrestricted control. Actions such as launching an app, finding an element, tapping, typing, swiping, pressing back, waiting, and reading the screen should be explicit tools with clear permission and safety boundaries.

Real progress rules

This idea will measure capability, not study time. A feature is only complete when there is evidence of:

Input -> System -> Real result

The first useful milestone is deliberately small: the Android app receives a command and opens another app through a real Android mechanism. From there, the system can move through a ladder of experiments: app launching, deterministic workflows, accessibility inspection, UI actions, observation, verification, AI planning, screen vision, voice, and eventually product-level reliability.

The working levels are: unknown, understood, reproduced, modified, integrated, reliable, and productized. Each level should be demonstrated rather than claimed.

First experiment

Build a native Android screen with a small set of real commands:

  • Open Browser
  • Open Maps
  • Open Calculator

Then record the result on a real device or emulator. The next experiment can make Calculator perform a short deterministic workflow, such as calculating 23 + 19 and verifying the result before introducing an AI model.

The full development context is documented in the Computer Use Agents planning conversation.