Microsoft Research Shows Off Motion Sensing Mechanical Keyboard, Detects Type–Hover–Swipe Gestures

The 2dn gen Microsoft Surface Touch Covers already supports gesture in them. But, you have to perform the gestures on the keyboard. This new project from Microsoft Research will allow you to perform gestures over the keyboard without even touching them. Such user experience is enabled by low-resolution matrix of infrared (IR) proximity sensors between the keys of the mechanical keyboard. Watch the video demo above.

We present a new type of augmented mechanical keyboard, sensing rich and expressive motion gestures performed both on and directly above the device. A low-resolution matrix of infrared (IR) proximity sensors is interspersed with the keys of a regular mechanical keyboard. This results in coarse but high frame-rate motion data. We extend a machine learning algorithm, traditionally used for static classification only, to robustly support dynamic, temporal gestures. We propose the use of motion signatures a technique that utilizes pairs of motion history images and a random forest classifier to robustly recognize a large set of motion gestures. Our technique achieves a mean per-frame classification accuracy of 75:6% in leave–one–subject–out and 89:9% in half-test/half-training cross-validation. We detail hardware and gesture recognition algorithm, provide accuracy results, and demonstrate a large set of gestures designed to be performed with the device. We conclude with qualitative feedback from users, discussion of limitations and areas for future work.

via: @h0xod

  • spydaweb

    Nice idea, but potentially frustrating if the keyboard picks up unintended gestures.

    • GG002

      That’s one of the bigger challenges of sensors in consumer space, but also one of the areas which has a lot of focus. Finding smart algorithms and/or a combination of various types of sensors for combination of data to detect whether something was intended or not.

      • spydaweb

        Yep, usually results in very limited gesture and scenario use.

  • alukard


  • Ben A

    I think its for the future windows, beyond next year windows 9. So Microsoft can combat a problem challenges of sensors. We possibly see this publicly in the next 18-24 months. I could be wrong, it could be sooner. who knows? GREAT JOB MS, keep on innovating and inventing new things that have never been done in technology.