Back to Robot Intelligence

Training data is only useful if users can see it, understand it, and act on it. This section covers the software backend that turns raw session data into performance insights, delivers AI-powered coaching analysis, and lets users control the robot conveniently from their phone, all without requiring internet access.

Three software layers connect the robot's ROS 2 backend, the desktop touchscreen GUI, and the phone dashboard:

Layer	Connects	How It Works
ROS Bridge	GUI ↔ ROS 2 Nodes	A background thread (QThread) subscribes to ROS topics and emits Qt signals, keeping the GUI responsive while receiving real-time data
IPC Files	Phone ↔ GUI	The phone dashboard writes JSON command files to /tmp/ which the GUI polls every 100ms, enabling remote control without direct network coupling
REST + WebSocket	Phone ↔ Server	HTTP API for data queries, WebSocket for live session updates (punch counts, round progress) pushed in real time

The phone dashboard exists because typing on a touchscreen with boxing gloves on is impractical, a friction first confirmed during the CDE AI Fair, so the phone provides a natural interface for browsing stats, chatting with the AI coach, and configuring training sessions.

Connecting from any phone, on any network

A normal localhost dashboard requires the phone and the robot to be on the same WiFi , in a gym, that's almost never true. BoxBunny solves this by giving every session a public URL via a localhost.run reverse tunnel, so a phone can open the dashboard from anywhere, even on cellular data, with no router configuration and no shared network.

If outbound internet isn't available the dashboard still serves directly over the robot's local WiFi access point as a fallback, the tunnel layer is opt-in.

The diagram below ties all three communication layers together, phone ↔ backend over HTTP/WebSocket, backend ↔ touchscreen GUI through /tmp/ IPC files, and GUI ↔ ROS 2 through the QThread bridge, showing exactly which messages flow where, and which database each side reads from.

Every training session generates a wealth of data, punch types, force levels, timing, movement patterns, defence outcomes. The analytics system organises this into four layers of insight, from individual sessions to long-term progress:

Insight Level	What Users See	Why It Matters
Session Summary	Punch distribution by type, force breakdown, rounds completed, punches per minute, max power, movement metrics	Immediate feedback after every session, what went well, what needs work
Trend Analysis	Time-series charts across 7 days, 30 days, 90 days, or all time for volume, reaction time, power, defence rate, and stamina	Track improvement over weeks and months, see if training is working
Peer Comparison	Population benchmarks segmented by age, gender, and skill level. Percentile tiers from "Getting Started" (<10th) to "Elite" (>90th)	Know where you stand relative to others at your level
Personal Records	All-time bests for power, stamina, and reaction time with date achieved	Motivation through personal milestones
Sparring Breakdown	Defence breakdown (blocks, slips, dodges, hits) and weakness profile by punch type	Identifies defensive gaps, feeds back into the AI sparring engine's targeting

The LLM coaching system (described in Section 5.3.2) surfaces on the phone dashboard through two features:

Post-Session Analysis

After each training session, the dashboard sends the full session summary to the LLM, which generates a 2–3 paragraph personalised analysis. The analysis references specific numbers from the session (e.g., "your cross power dropped 15% in round 3") and can suggest specific drills to address weaknesses. These suggestions appear as tappable action cards, one tap loads the drill configuration and starts it on the robot.

Interactive Chat

A full-screen chat interface lets users have ongoing conversations with the AI coach. Responses stream word-by-word (20ms per word) for a natural feel. Users ask about technique ("how do I improve my left hook?"), request custom training plans, or get explanations of their statistics. Conversation history is persisted across sessions, and follow-up suggestions appear after each response.

This was directly motivated by CDE AI Fair feedback, users wanted to interact with the coach but typing on the 7-inch touchscreen was awkward. The phone dashboard solves this with a natural mobile chat experience.

The frontend stack (Vue 3 + Pinia + Tailwind + Chart.js) and the 5 user-facing views are introduced in 5.1 Implementation. Two of those views, AI Coach Chat and Training Remote , are the most technically interesting on the backend side because they cross every layer of the stack (HTTP → FastAPI → ROS 2 → touchscreen GUI). Both run over the robot's own WiFi access point with no external network required.

AI Coach Chat, Phone → FastAPI → ROS → LLM

A single chat message crosses every layer of the stack and streams back word-by-word. The diagram below traces the full path:

Training Remote, IPC-File Bridge to the Touchscreen GUI

The phone never talks to ROS directly. Instead, the FastAPI backend writes JSON command files into /tmp/ and the touchscreen GUI polls them on a 100ms tick:

• Discrete commands (navigate page, start session, browse presets) , backend writes one file, GUI reads it once and executes.
• Press-and-hold height control, the only command that needs continuous input. While the user holds the up/down button, the backend re-writes /tmp/boxbunny_height_cmd.json every ~100ms; the GUI re-reads it on the same tick and drives the height motor.
• Stale-file deadman, if the height command file ever gets older than 500ms (finger released, network blip, backend stalls), the GUI auto-stops the motor. The robot can never run away because no one is checking, which is what makes press-and-hold safe to expose to a remote phone.

The end result: users can configure a full training session from their phone, walk over to the robot with gloves already on, and start training immediately, no typing on the touchscreen.

Full Dashboard Walkthrough

The video below shows all dashboard views in action:

The phone dashboard communicates with the robot through a FastAPI backend running on the Jetson. Seven API routers handle different aspects of the system:

Router	Purpose	Key Detail
auth	Login, signup, profile management	JWT tokens (7-day expiry) + SHA-256 pattern lock for gloved-hand auth
sessions	Session history, trends, detail views	Paginated with mode filtering, raw sensor data export
gamification	XP, ranks, achievements, benchmarks	Population percentile comparison by demographics
chat	AI coach messaging	Bridges to LLM ROS service, parses drill action tags
remote	GUI remote control, height adjustment	Writes IPC command files for touchscreen navigation
presets	Training preset management	Create, edit, favourite, track usage count
coach	Group training management	Coach role required, station control, live participant stats

Two-Tier Database

Data is stored in SQLite across two tiers, designed for privacy and scalability:

Splitting per-user data into its own file gives complete data isolation (one user cannot read another's data even at the file level), avoids write-lock contention during concurrent training on multiple stations, and makes per-user export, backup, or deletion a single file operation.

Real-Time Updates

During active training, the phone dashboard receives live updates via WebSocket, punch counts, round progress, and session state changes are pushed instantly rather than requiring the user to refresh. The connection uses 30-second keepalive pings and automatically reconnects with exponential backoff (up to 10 attempts) if dropped. On reconnect, the server sends the last known state so the dashboard catches up without missing data.

The gamification layer, 6 rank tiers, 12+ achievement badges, daily streaks, and personal records, is introduced in 5.1 Implementation. On the backend, all of it lives in the per-user database and is computed by the gamification router after every session through a single deterministic formula:

xp_earned = max(
 base_xp[mode] # 30–80 by training mode
 * completion_pct # 0.0–1.0 of rounds finished
 * score_multiplier # scales with session score
 * difficulty_multiplier # 1.0× beginner → 2.0× elite
 + streak_bonus, # active daily streak → bonus
 10 # minimum 10 XP per session
)

The same router also exposes the read-side endpoints used by both the phone and the touchscreen GUI:

Each gamification dimension has its own update rule, all running on the per-user database with no client-side state:

The screenshots below show what users see on the phone: the home view surfaces XP progress, current rank, and the daily streak counter, while the analytics view shows the trend charts, personal records, and peer percentile rankings powered by the same gamification router.