New Kinect Can Read Human Sign-Language With Ease

11

Microsoft Kinect is creating quite a revolution in the field of Natural User Interface. The low cost 3D depth camera like Kinect opens up infinite possibilities in our computing world. Sign language is the primary language for many deaf and hard-of-hearing people, but currently it is not possible for these people to interact with computers using their native language. Microsoft Research Asia has developed a system using Kinect which can detect sign languages used by these people and it works well.

Here are some info on the project,

Kinect, with its ability to provide depth information and color data simultaneously, makes it easier to track hand and body actions more accurately—and quickly.

In this project—which is being shown during the DemoFest portion of Faculty Summit 2013, which brings more than 400 academic researchers to Microsoft headquarters to share insight into impactful research—the hand tracking leads to a process of 3-D motion-trajectory alignment and matching for individual words in sign language. The words are generated via hand tracking by the Kinect for Windows software and then normalized, and matching scores are computed to identify the most relevant candidates when a signed word is analyzed.

The algorithm for this 3-D trajectory matching, in turn, has enabled the construction of a system for sign-language recognition and translation, consisting of two modes. The first, Translation Mode, translates sign language into text or speech. The technology currently supports American sign language but has potential for all varieties of sign language.

The second, Communications Mode, enables communications between a hearing person and a deaf or hard-of-hearing person by use of an avatar. Guided by text input from a keyboard, the avatar can display the corresponding sign-language sentence. The deaf or hard-of-hearing person responds using sign language, and the system converts that answer into text.

Read more from the link below.

Source: Microsoft



About Author

Pradeep, a Computer Science & Engineering graduate.

  • donzebe

    Wow! what a break thru

    • Gromanon

      for a entertainment/gaming console? Hail Yeah it is!

  • BIAS

    I wus gonna get the One even before this.

  • jimski27

    Never though of Kinect for this kind of application. Very impressive.

    • Robin Visser

      You should see what they do with it in the field of medicine. It’s awesome.

  • emblaze

    this is the old Kinect for windows. not the new one.

    • Yuan Taizong

      This means that the new one can only get better.

  • Yuan Taizong

    Microsoft, Masters of innovation :-D

  • Yuan Taizong

    Oh great, now deaf people won’t have privacy from the spying eye, either. (N.S.A. joke).

  • Robin Visser

    Microsoft are truly the masters of making software and hardware that is actually useful. I’d love to look around their R&D dept. Like many have said before, the Xbox One will have a much wider audience than the competition due to its extensive feature package.

  • XBoXer

    Why is it so slow? It would be faster to type the conversation.