Enabling Interaction on Everyday Surfaces

Computers are now ubiquitous. However, computers and digital content have remained largely separate from the physical world – users explicitly interact with computers through small screens and input devices, and the “virtual world” of digital content has had very little overlap with the immediate, physical world. My work aims to help computing escape the confines of screens and devices, and spill information- and computationally-rich digital content out into the familiar world around us. I approach this problem from several directions: from the low-level algorithmic work of providing ad hoc touch sensing on everyday surfaces, to high-level questions surrounding the interaction design between physical and virtual realms. I have built many embodiments of these mixed-reality experiences, including a computational lightbulb capable of projecting interactive content onto everyday surfaces, as well as a head-mounted augmented reality system that integrates touch interaction on the environment.