> ## Documentation Index
> Fetch the complete documentation index at: https://docs.uos.agency/llms.txt
> Use this file to discover all available pages before exploring further.

# uGAME

> Watch an AI agent play Game Boy games inside uOS

export const AgentLoopDiagram = () => <pre className="ascii-diagram">{`┌─────────────────────────────────────────────────────────────┐
│                  uGAME AGENT LOOP (~1 Hz)                    │
└─────────────────────────────────────────────────────────────┘

   ┌──────────────────┐
   │  Capture screen  │   canvas.toDataURL() from the emulator
   └────────┬─────────┘
            ▼
   ┌──────────────────┐
   │  Read text/state │   Claude Opus vision → on-screen text
   │  (Claude vision) │   + optional RAM reads (position, party)
   └────────┬─────────┘
            ▼
   ┌──────────────────┐
   │  Decide action   │   rule-based heuristics per screen type
   │  (agent-core)    │   (Overworld / Battle / Menu / Dialogue)
   └────────┬─────────┘
            ▼
   ┌──────────────────┐
   │  Press button    │   synthetic keyboard events → emulator
   └──────────────────┘`}</pre>;

<div className="app-doc-header">
  <img src="https://mintcdn.com/uos-9a805c5e/yMf8CRCNR32nX9jN/assets/ugame-icon.png?fit=max&auto=format&n=yMf8CRCNR32nX9jN&q=85&s=d5623a6bc7091881f9b68f9b38b42165" alt="uGAME" className="app-doc-logo" width="2250" height="2250" data-path="assets/ugame-icon.png" />
</div>

uGAME runs a Game Boy emulator inside uOS and lets the uOS Assistant play for you. Today it's set up for **Pokémon**, and you can either play manually with the on-screen controls (or your keyboard) or hand control to the agent and watch it think and act in real time.

<Note>
  uGAME wraps the open-source [GameBoy-Online](https://github.com/taisel/GameBoy-Online) emulator by Grant Galitz. Your progress can be saved to the cloud when your wallet is connected.
</Note>

## How AI play works

When you ask the Assistant to play (for example, "play pokemon"), uGAME opens and the agent starts a loop roughly once per second:

<AgentLoopDiagram />

Each cycle it:

1. **Captures the screen**: grabs a frame from the emulator canvas.
2. **Reads the state**: sends the frame to Claude (vision) to extract on-screen text, and optionally reads Game Boy RAM for things like the player's position and party.
3. **Decides an action**: a rule-based engine picks a move based on the current screen type (overworld, battle, menu, or dialogue).
4. **Presses a button**: sends a synthetic key press to the emulator, just like a human would.

<Tip>
  In **Agent Mode**, the Assistant streams its reasoning (what it sees, where it thinks it is, and what it's doing next) so you can follow along.
</Tip>

## What you can ask

The Assistant recognizes gameplay requests and opens uGAME for you. Useful prompts:

* **Start playing**: "play pokemon", "play the game", "explore"
* **Stop**: "stop" ends the current task and turns off Agent Mode

Exploration is best-effort: the agent moves around, advances dialogue, and handles menus and battles with built-in heuristics. It's a fun, transparent demo of an agent driving a real app, not a speedrunning bot.

## Good to know

* **Game**: Currently tuned for Pokémon (Game Boy). Other titles aren't wired up yet.
* **Decision-making**: Rule-based today. Vision is used to read the screen, not to choose every button.
* **Saves**: Cloud save/restore is available when your wallet is connected.

<Note>
  Pathfinding to named locations, item-hunting, and full task planning are on the roadmap. The current agent focuses on perceive → decide → act with simple, readable rules.
</Note>