General

Minecraft, with an AI companion beside me

A fun build that became a released Fabric mod: a companion with a real body, real tools, and a plan for helping in my single-player world.

Oct 22, 2026 - 6 min read

• Fun
• Minecraft
• AI agents
• Fabric
• Released
• Build story

I wanted an AI companion in Minecraft. Someone who could walk beside me, notice trouble, gather materials, and help with a task while I carried on playing.

The fun part of the idea was giving an agent a place in a world I already enjoy. A request like “gather some logs” could become movement through terrain, a tool being used, items being picked up, and a companion returning with something useful.

That became Minecraft Companion Mod, a released Fabric project for Minecraft Java 26.3. The current release covered here is 0.2.0.

A companion that shares the world

The starting request, recorded in my October 21 Codex session, was quite specific. The companion needed a player-like shape, a skin I would provide, an API connection to a model, and the ability to interact with blocks, tools, and other entities.

I also wanted two modes. In survival, the companion should live with health, hunger, equipment, durability, and actual materials. In invincible, it could survive hazards, fly, and create requested work supplies. The choice belongs to the player.

One point became especially important during the build: I should still be the live player in my single-player world. The companion is a separate entity alongside me. It does not take over my controls, require another account, or need an external multiplayer server.

Giving the model decisions, and the game responsibility

The implementation uses a loop of observing the world, choosing an action, validating it, doing the work, and checking what changed.

The model sees structured observations about nearby terrain, entities, inventory, and the task. It can call tools, but the game checks the requests before changing anything. Mining needs reachable blocks and suitable tools. Crafting needs materials. Storage delivery needs space. A successful resource job must be supported by actual inventory or completed delivery.

Following and immediate defense run locally. That matters because API calls take time: a companion should not wait for a remote response before reacting to an immediate threat. Native Gemini and OpenAI-compatible endpoints provide the model connection; a compatible server needs tool-calling support to control gameplay.

“I don’t see any visual movements”

The most useful feedback in the first session was my own report that I could see chat messages describing actions, but no visible movement.

Earlier checks had passed, yet that did not settle what the experience looked like from the player’s perspective. When the session ended, the normal-speed movement check was still unfinished.

The continuation returned to that exact concern. A paced recording showed the companion walking 11.1 blocks, moving its legs, swinging a pickaxe, and mining with real tool wear. My supplied skin was also checked in the game.

That moment shaped the project. If I ask for help in Minecraft, I want to see the companion doing it. Chat output and a successful build alone cannot establish that.

The settings needed to feel like part of the game

Once I started using the mod, setup became the next improvement. I wanted a companion button in Minecraft’s menus, with the API, skin, arm size, and model settings together.

Version 0.1.1 added that experience. Options → AI Companion opens the settings from the title screen or inside a world, and the pause menu has its own entry. Connection, Appearance, and Limits tabs handle the provider, endpoint, model, masked API key, connection test, skin import, and standard or slim arms.

This is a small product lesson I like: a working capability becomes much easier to use when its controls live where the player expects them.

Helping means planning several steps

A single action is useful, but a goal usually needs several. Crafting a pickaxe may require gathering logs, making planks, making sticks, and crafting the tool. Reaching a high log might require building a way up first.

The 0.2.0 update added plans of up to 16 steps, executed in order through the same local checks. When something fails, the remaining steps stop. The result goes back to the model so it can reconsider the situation. Checkpoints preserve completed work and delivery accounting across a paused task.

The companion can also build supported access with blocks and excavate descending stairs. In survival, this uses real supplies and keeps ordinary gravity. It refuses unsafe access rather than treating every destination as automatically reachable.

I then asked for a much larger travel allowance, with the distance editable in settings. Permitted trips now default to 1,000,000 blocks. The Limits tab can change that, and 0 allows unlimited distance within the world border. Allow trip is still an explicit player choice, and task time, action, and request budgets remain in effect.

What was actually checked

The project’s validation record separates gameplay, visual checks, and live model behavior. For 0.2.0 it reports:

• 19 unit/provider checks and 34 gameplay scenarios.
• A client check covering the settings, saved trip range, controls, and world reopening.
• An actual 1,024-block independent trip into initially unloaded chunks, with chunk-loading cleanup.
• A paced elevated-log job showing building, climbing, mining, descending, and collection with supplied tools and blocks.
• A separate live Gemini run that started with an empty inventory, two reachable logs, and a crafting table, then completed a five-step plan for a wooden pickaxe in about 16 seconds.

Those fixtures have limits. The movement recording used the direct validated action executor; it demonstrates physical actions separately from model planning. The live planning run used prepared surroundings and an extended headless deadline. The million-block setting is not proof of a million-block journey.

The original release also recorded a 30-minute mixed gameplay soak and successful native and compatible API adapter checks. Both adapters used the same hosted model, so those results do not compare two different models. They are earlier functional evidence, not new 0.2.0 soak results or a model benchmark.

Try it, and tell me what happens

The project is released under Fun in my project collection. There is still plenty to improve: planning through less predictable terrain, recovery when a goal gets complicated, and player mechanics beyond the first-release scope.

For setup, use the Minecraft Companion Mod repository. This release targets Minecraft Java 26.3, Fabric Loader 0.19.5, Fabric API 0.164.0+26.3, and Java 25. Put the main companion JAR and Fabric API in the profile’s mods folder. The sources JAR is for reading the code. Updates require replacing the old companion JAR and restarting Minecraft.

Model behavior varies with the chosen provider, tool support, and latency. Gameplay observations and recent task context go to the configured endpoint; the mod’s documentation describes the configuration and data boundary. Dedicated and public servers are outside this release’s scope.

If you try it, I would especially like feedback about what you saw the companion actually do, where it got stuck, and which controls felt confusing. Those observations are how this becomes a better companion to play alongside.

Read the project build story for the release progression. This article was reconstructed from the October 21–22 Codex build sessions and cross-checked against the project’s README, architecture notes, and validation record. Minecraft belongs to Mojang and Microsoft; this is an independent companion mod.