Computer Vision Controlled Roomba

Way back in 2012, I had the opportunity take a project to the Portland Mini MakerFaire. I had a Roomba, and a Microsoft Kinect and was very excited about the possibility of making the Roomba drive around by letting people control it just with motion. I refined that project a couple of times over the next few years. The most successful effort was at OMSI After dark in April of 2015.

The full stack was this:

  • MacBook Pro connected to a 1st Gen Microsoft Kinect
  • A Python app reading the depth camera data, and packaging and sending the
  • The Skeleton Jelly Library and the underlying OpenNI library (this one was a bear because it needed to be compiled for MacOS X)
  • A Roving Networks Bluetooth adapter wired to an old Mac serial cable – a Mini DIN 8 connector that the Roomba exposed.

When running the app, the bluetooth would connect to the BT module on the Roomba, and send the command bytes to put the Roomba into dev mode. From there the app would send differential drive motor PWM values to the Roomba.