Voxta Minecraft Companion
Voxta's open-source desktop app that drops an AI character into your Minecraft world — voice chat, autonomous crafting, building, combat, and full world perception.
Voxta Minecraft Companion is an Electron desktop app that connects a running Voxta Server to a Minecraft Java server. The bot joins your world, perceives the game state in real time, talks to you over voice or text, and acts on the world based on the conversation — crafting, building, fighting, fishing, riding horses.
It's a community-built showcase of how far Voxta's WebSocket API lets you go: every action below is driven by Voxta's action inference firing on top of normal AI dialogue.
What the companion does
AI & conversation
- Real-time world perception — the AI sees health, hunger, biome, time of day, weather, nearby entities, inventory, and shelter status as live context.
- Voice chat — talk to the bot through your microphone via Voxta's STT.
- Action inference — the AI decides what to do based on the conversation, either before or after replying.
- Speech interruption — urgent events (damage, explosions) cut the bot off mid-sentence.
- Voice-chance sliders — per-category probability that a result triggers a voiced reply vs. a silent context note.
Autonomous crafting
- Full chain from nothing — "Craft me a sword" with an empty inventory: bot chops trees → planks → crafting table → place → craft → pick the table back up.
- Recursive dependencies — auto-crafts intermediate materials in order.
- Smart material selection — picks the best variant (oak / spruce / birch / …) based on inventory.
- Paper-server compatible — proper window-close sequence prevents inventory rollback issues.
Building
- Built-in structures: Shelter (7×7 hut), Watchtower (5×5 tower with stairs), Wall (3×3 defensive barrier with arrow slit).
- Auto-orientation — walls placed ahead of the player, rotated to match their look direction.
- Material-aware — cobblestone > planks > dirt, reports block counts needed.
- Custom blueprints — load extra structures from JSON files.
- Build guard — every other AI action blocks during construction (except
mc_stop).
Combat
- Auto-defense against hostile mobs in range.
- Modes: Aggro (attack everything hostile), Hunt (target farm animals for food), Guard (patrol + defend an area), Passive (only fights when attacked).
- Hit-and-run kiting against ranged attackers.
- Creeper-explosion detection via packet listener avoids false "defeated" reports.
- Combat timeouts cap fights at 60s so the bot never loops forever.
Mounted navigation
- Mount and steer horses autonomously; follows at gallop speed.
- Rides boats as a passenger.
- Auto-dismounts before attacking, following on foot, or going indoors.
- Vehicle-aware pathfinding — watchdog and spatial audio use vehicle position while mounted.
Survival & inventory
- Fishing, smelting, auto-eat, swim-jumping when submerged.
- Give / receive items by tossing.
- Chest interactions (store, take).
- Place blocks from inventory.
- Sleep in beds at night.
Vision
- Screen capture — screenshots of your Minecraft window sent to Voxta's vision pipeline.
- Eyes mode — capture from the bot's spectator camera for true "bot vision".
How it fits with Voxta
The companion is a client of your Voxta Server, same as Voxy or the VAM plugin — it connects to the SignalR / WebSocket hub at 127.0.0.1:5384 and consumes the action-inference API. Voxta does the heavy lifting (LLM, TTS, STT, character state), the companion handles all the Minecraft-side execution.
That means everything you've configured in Voxta — your character, your LLM, your voice, your scripts — comes along when the companion connects. You can run the same character in chat, on the desktop with Voxy, in Virt-A-Mate, and in Minecraft, with the same persona reacting to each environment differently.
Open source
The companion is MIT-licensed and lives on GitHub at voxta-ai/voxta-minecraft-companion. PRs, blueprints, action contributions all welcome.