Back to notes

During my PhD research I ran a lot of reinforcement learning experiments. At some point I got tired of staring at terminal logs and decided to build a proper interface for it. Dash and Plotly turned out to be a surprisingly good fit, within a few hours I had live reward curves, algorithm selection, toggles for observation and reward shaping, all running in the browser. I was genuinely impressed by how much you can build with those two libraries.

The interface had a natural two-panel structure: controls on one side, monitoring on the other. But recently I got an idea and asked Claude to make it look like a Nintendo DS.

Try it — click the cartridge slot, select an algorithm, and hit TRAIN.

The DS form factor maps surprisingly well onto the RL workflow. Top screen for monitoring, bottom screen for configuration, cartridge slot for loading environments. The whole shell is just CSS, which gives you full control over the plastic texture, bezels, speaker grilles, and all the rest.

I liked the result enough to clean it up and put it on GitHub. It supports standard Gymnasium environments, custom envs loaded from a file path, custom SB3-compatible algorithm classes, TensorBoard, and optional Weights & Biases logging. Most useful if you are already in the SB3 + Gymnasium ecosystem and want something nicer than a terminal.