Projects

Q-learning from Frozen Lake, scaled up to raw pixels with a convolutional network, a replay buffer and a target network, following the DQN paper. The agent beats the human reference score after one million steps.

Reinforcement learning · DQN · PyTorch · Gymnasium · Atari

A low-cost SO-101 arm with millimetres of play learns a sub-millimetre task, plugging an SFP+ cable into a network card, from demonstrations, ACT and DAgger.

Imitation learning · ACT · DAgger · LeRobot · PyTorch · SO-101

Working through Sutton and Barto with small experiments, a 10-armed bandit and tabular Q-learning on Frozen Lake, including an interactive demo.

Reinforcement learning · Q-learning · Python · Gymnasium · NumPy

From a formula to a triangle mesh. A Java implementation of marching cubes, explained step by step with an interactive 2D version of the algorithm.

Java · Computer graphics · Geometry

How a 3D model becomes pixels. A Java/Swing renderer that takes OBJ meshes through every stage of the rendering pipeline by hand, with an interactive version to step through.

Java · Computer graphics

A GPU fragment shader, run on the CPU in plain C++. Vector types that mimic GLSL, a nested loop over the pixels and raw PPM files, with no graphics API and no image library. Includes an interactive version to trace a single pixel.

C++ · Computer graphics

Finding a flying object in 3D from several ordinary 2D cameras. Moving pixels become rays, the rays are walked through a sparse voxel grid, and the voxels where rays from different cameras meet mark the object.

C++ · Computer vision · Eigen · Blender

A plate on three stepper motors that should keep a table tennis ball in its centre, controlled only through a camera. A PPO policy learns the task from pixels in Isaac Lab, and a C++ pipeline on a Raspberry Pi runs it on real camera images. It works in simulation, the transfer to the hardware is not finished.

Reinforcement learning · Isaac Lab · PPO · C++ · ONNX · Raspberry Pi

A machine that sorts pills by colour, built in a team of three on an MSP430. Event-driven firmware in C with a colour sensor, two servos and an LCD on one I2C bus, and an interactive model of its state machine.

C · MSP430 · I2C · Embedded

YOLOv11n running client-side on the live webcam feed with ONNX Runtime Web and React, from camera frame to bounding boxes.

YOLO · ONNX Runtime Web · React · TypeScript

A robot arm that copies a small hand-held replica of itself. Four potentiometers, an MSP430 and a PCA9685 servo driver, with interrupt-driven ADC and I2C firmware in plain C, and an interactive model of the signal chain from knob to servo.

C · MSP430 · I2C · Embedded

A small C++ and SFML program where a draggable sun sends out 360 rays. Shadows are never computed, they appear wherever no ray arrives.

C++ · SFML · Computer graphics

A neural network written from scratch in C++, without any ML library. Forward pass, backpropagation and training on MNIST explained step by step, with the trained network running in the browser.

C++ · Neural networks · MNIST · OpenCV