Hunter Wumpus

What it is

Hunt the Wumpus, inverted. On a 10×10 grid you search for gold, or for the Wumpus itself, through fog of war, reading three sensory hints: a Breeze near pits, a Stench near the Wumpus, a Shine near gold. The twist is the monster. It isn't scripted. It's a PPO-trained reinforcement-learning agent that hunts you, and it tracks you from the fading scent you leave behind rather than from your exact coordinates. Four ways it ends: you find the gold and escape, you put your one arrow through the Wumpus, you fall into a pit, or it catches you in the dark.

Architecture

A React + Vite frontend talks to a FastAPI backend over a small REST API. The backend owns a deterministic game engine and hands the enemy's turn to the trained agent.

• Client: the grid, tiles, HUD, and a useControls keyboard hook, rendering only the fog-of-war state the server returns.
• Server: a deterministic engine validates each move, updates the board and the scent memory, then asks the agent for the Wumpus's move.
• Agent: a stable-baselines3 PPO policy, loaded per difficulty tier, with a random-Wumpus fallback if a model is missing so the game never hard-fails.

How the Wumpus thinks

This is the part I care about. A monster that always knows exactly where you are is either trivial or unfair, so the Wumpus never sees your coordinates directly. It sees a 9-dimensional observation: its own normalized position, yours, and five local scent readings, one for its tile and one for each cardinal neighbour.

You leave scent on the tiles you walk, and it decays to nothing over three steps (MAX_SCENT = 3). The agent learns to climb that gradient, which reads as hunting: it follows where you went, loses the trail when it goes cold, and re-acquires it when you move again. Emergent tracking, never a coordinate lookup.

Because every position in the observation is normalized to [0, 1], the same policy transfers from the small grid it trains on to the full 10×10 board it plays on.

Training

The Wumpus is the only thing learning. During training the player just moves at random, and the agent optimizes a shaped reward with PPO:

The step penalty pushes for efficiency, the scent bonus rewards actually following the trail, and the terminal values make catching you the whole point. The rest is standard Stable-Baselines3 PPO:

An EvalCallback scores the policy every 10,000 steps and keeps the best checkpoint. A model only ships if it beats a random-Wumpus baseline by at least 20 mean reward, otherwise it falls back to random. Here is the loop:

Difficulty tiers

Each difficulty is a separately trained policy. The lever is the training budget: more steps, a sharper hunter.

The API

Three endpoints, JSON in and out:

• POST /game/start opens a session and returns the first state: grid size, your position, the senses, arrows remaining.
• POST /game/move applies your action (NORTH through WEST, or SHOOT_*), advances the Wumpus, and returns the new state.
• GET /game/{id}/status re-reads the current state, for a refresh or a reconnect.

Sessions live in memory, which is fine for one player at a time. Persistence and websockets are on the list.

The research

This became a published paper, Reinforcement Learning for Constraint Satisfaction Game Agents, co-authored with Gagan Venkat (DOI 10.5281/zenodo.20076630). It works up from tabular methods, through deep Q-networks, to the adversarial PPO agent. More on the research page.

What I'd do differently

Training an agent to be fun to lose to is a different objective than training it to win. The interesting work lived in the reward shaping and the scent model, not the algorithm. The honest limitations: it trains on a smaller grid than it plays on, there is a single Wumpus, the player never learns, and sessions live in memory. Self-play, multiple Wumpuses, and a persistent backend are the obvious next steps.

Play it

Play Hunter Wumpus →. Pick a difficulty, each tier a differently-trained Wumpus, and see how long you last.