Analysis and interpretation: debugging why your bot AI lost

Your football team loses 3–0. On the drive home someone says "we were rubbish", and everyone agrees. Next week you lose again, because "we were rubbish" doesn't tell anyone what to change. Was it the passing? The defending on corners? The first ten minutes, or the last?

The same thing happens when a bot loses a match. The tempting conclusion is "my code is bad", which is just as useless. This post is about doing better: taking a result apart until it tells you something.

What analysis and interpretation are

Analysis means breaking something into its parts and looking at each one. Interpretation means working out what the evidence actually says, and not what you assumed before you looked. Together they're part of what's usually called critical thinking, and they lean on executive functions such as holding several facts in mind at once and switching between them.

The skill isn't knowing the answer. It's asking smaller questions, each of which can be answered. Debugging a bot's AI in a programming game is steady practice in it: every lost match is evidence waiting to be read.

Break the match into parts

Every bot does four jobs, every tick. When it loses, one or more of them failed, and each leaves its own evidence:

Part	The question	Where to look
Moving	Did it go where it meant to, or get stuck?	The replay, stepped tick by tick
Sensing	Did it see the enemy? Was the scanner pointing the right way?	Scanner arcs in the Arena
Deciding	What did it think was happening?	Its own log lines in Bot logs
Firing	Did its shots hit the right thing?	Events, and the damage lines in Bot logs

The Arena lets you step through any match one tick at a time (← and →), and clicking a line in Events replays the moment just before it at half speed. The FAQ covers the replay controls and finding out why your bot lost.

But the replay only shows what the bot did. To see what it thought, you have to ask it.

Debugging: make the bot tell you what it saw

log(...) writes a line to the bot's log. It costs no CPU cycles and never changes the match, so you can log as much as you like (up to 8 lines a tick). Each {} shows a number; {|} shows | for true and . for false. Here's a bot that stands still, sweeps its scanner and fires at whatever it finds, logging one line per look:

from bot import Chassis, Scanner, Laser, wait

chassis = Chassis(0)
scanner = Scanner(1)
laser = Laser(2)

look = 0  # where the scanner's arc points: a field bearing, in degrees
last = 0  # the contact's bearing at the last look

while True:
    wait()  # commands out, next tick, fresh readings in

    if scanner.any:
        b = scanner.bearing(0)
        # One line per look: what it saw, how far it moved, what we did.
        log("contact @ {}, moved {}, fire {|}, hull {}", b, b - last, laser.ready, chassis.hull)
        last = b
        look = b
        if laser.ready:
            laser.fire(b)
    else:
        look += 30  # nothing: sweep on, one arc a tick
        if look >= 360:
            look -= 360

    scanner.sense = look

In one of our test runs, its log began like this:

contact @ 90, moved 90, fire |, hull 100
contact @ 90, moved 0, fire ., hull 100
contact @ 90, moved 0, fire ., hull 100
contact @ 90, moved 0, fire ., hull 100

Interpret the numbers

Numbers only mean something once you know what they measure. So go through them one by one:

• contact @ 90: a scanner bearing is a direction in degrees on the field: 0 is east, and positive turns left, so 90 is north. The scanner gives no distance and can't tell a bot from a wall.
• moved 90 on the first line: this looks dramatic, but it's not movement at all. last starts at 0, so the first look's "moved" is just the bearing itself. Knowing how a number was produced stops you reading meaning into it.
• moved 0 after that: our bot is standing still, and the contact is too. Tick after tick, exactly the same bearing.
• fire |, then .: the laser fired, then went into its cooldown. After each shot it needs 10 ticks (half a second) before laser.ready is true again.

Put together, these say something specific. A contact at exactly 90, that never moves, is very likely a wall. Walls and the faces of blocks line up with the field, so they ping at almost exactly 0, 90, −90 or −180. Our bot isn't fighting an enemy. It's shooting a wall, every half second, for the whole match.

That's a far more useful conclusion than "my code is bad". It points to the fix: before firing, check whether a contact has moved. The FAQ's scanner guide has other ways to tell friend, foe and wall apart.

Look for the pattern

Some problems only show up as a pattern across many lines. In another run, a bot of ours lost hull steadily while no enemy was anywhere near it. So we changed what it logged: when the hull drops, how long since its own last shot?

from bot import Chassis, Scanner, Laser, cpu, wait

chassis = Chassis(0)
scanner = Scanner(1)
laser = Laser(2)

look = 0       # where the scanner's arc points
hull = 100     # the hull at the last look
shot = -100    # the tick of our last shot

while True:
    wait()

    # Damage: did the hull drop since last tick? How long since we fired?
    if chassis.hull < hull:
        log("hit: hull {}, {} ticks after my last shot", chassis.hull, cpu.tick - shot)
    hull = chassis.hull

    if scanner.any:
        b = scanner.bearing(0)
        look = b
        if laser.ready:
            laser.fire(b)
            shot = cpu.tick
            log("fire @ {}", b)
    else:
        look += 30
        if look >= 360:
            look -= 360

    scanner.sense = look

Logging only when something happens (a shot, a hit) makes the log read like a story instead of a wall of numbers. This one read:

fire @ -90.01
hit: hull 90, 2 ticks after my last shot
fire @ -90.01
hit: hull 80, 2 ticks after my last shot
fire @ -90.01
hit: hull 70, 2 ticks after my last shot

and so on, down to 0. Every hit came exactly 2 ticks after one of its own shots, at a bearing a hair from −90. That isn't an enemy. A laser shot does 10 damage to everything within 0.78 m of where the beam stops, and this bot was firing at a wall it was right up against, so it was inside its own blast. It destroyed itself in under five seconds. Bot logs also lists every hit's source, and a hit like this one is marked own blast.

One line on its own proves nothing. Ten lines with the same gap tell you the cause.

The same habit away from the screen

• A bad test result. Not "I'm bad at maths" but: which questions did I lose marks on? Were they all one topic, or all at the end, when time ran out? Those are different problems with different fixes.
• Money that disappears. Write down a week's spending and group it. The pattern (lunches out, small online orders) is the finding, not the total.
• A project at work that slipped. Split it into stages and find where the time went. Often it was one stage, and often it was waiting, not working.
• Sport. Watch the replay of the goals you let in. Did they come from the same side, or the same kind of move?
• An argument that keeps coming back. What sets it off, and when? Patterns in people are slower to spot, but the method is the same: collect what happened before you decide what it means.

Two warnings carry over from bots. First, know what your numbers measure before you trust them, like the first moved 90. Second, look for the pattern before you name the cause: one hit could be anything; ten, each 2 ticks after a shot, is evidence.

Try it

1. Write the first program above (on a design with a scanner on the sensor mount and a laser on the tool mount), save it, and in the Test arena tab place your bot near a block, with another bot to face. Click Run test. The FAQ explains how to see what your program is doing.
2. Read Bot logs. For each contact, decide: wall, teammate or enemy? Write down which numbers told you.
3. Step through the replay with Scanner arcs on and check your answers against what you see.
4. Add one log line that would have answered your hardest question, and run it again.

The Coding samples tab in Brain code has a sample on keeping time and logging, among others, under Basics.

A loss is a pile of evidence. Take it apart, and it will usually tell you what to do next.