Microsoft Research Goes Behind The Scenes About Bing Platform That Powers Cortana

Bing New Logo

Microsoft revealed the Cortana digital assistant platform at BUILD developer conference earlier this month. Behind the scenes, its Microsoft’s Bing platform that powers Cortana. Even though Cortana responds based on your intent and behavior now, MSR expects it to interact in an increasingly anticipatory, natural manner in the future. Microsoft Research detailed some of the technologies involved in building Cortana. Even though many of us would just categorize them under Artificial Intelligence label, Microsoft Research has detailed all of those technologies involved.

  • Speech recognition,
  • Semantic/natural language processing,
  • Dialogue modeling between human and machines,
  • Spoken-language generation,

Microsoft Research has used all of the above base technologies for a creating a virtual personal assistant based on the work they have done in different areas of personal-assistant technology.

Cortana’s design philosophy is therefore entrenched in state-of-the-art machine-learning and data-mining algorithms. Furthermore both developers and researchers are able to use Microsoft’s broad assets across commercial and enterprise products, including strong ties to Bing web search and Microsoft speech algorithms and data.

If Heck has set the bar high for Cortana’s future, it’s because of the deep, varied expertise within Microsoft Research.

“Microsoft Research has a long and broad history in AI,” he says. “There are leading scientists and pioneers in the AI field who work here. The underlying vision for this work and where it can go was derived from Eric Horvitz’s work on conversational interactions and understanding, which go as far back as the early ’90s. Speech and natural language processing are research areas of long standing, and so is machine learning. Plus, Microsoft Research is a leader in deep-learning and deep-neural-network research.”

What you saw with Cortana in Windows Phone 8.1 is just the beginning of the years of journey.

“Microsoft has intentionally built Cortana to scale out to all the different domains,” Heck says. “Having a long-term vision means we have a long-term architecture. The goal is to support all types of human interaction—whether it’s speech, text, or gestures—across domains of information and function and make it as easy as a natural conversation.”

Read about it in full detail here.

  • whatup12

    I really see the overarching long term strategy here–large information infrastructure powering more user-focused MSFT products. If they ditched bing, they would lose so much control over that vision and just don’t see it happening. Never mind bing may end up being more profitable given some of the recent sales data, that would just be icing on the cake. the cake itself is that bing represents access to information as the foundation of the future of the company.

    • NiXuqis

      I think Microsoft has finally recognized that the best way forward is to create cross platform services and software. Nadella understands Apple has the high end and partnering with them supplying service is a great step in this direction. At the same time he will give way WP8 and W8 to bleed some of the low end from Android. Though the primary focus is on Azure and enterprise services.

      • whatup12

        Agree–every company needs to have its bread and butter while it future-proofs itself. Google has search allowing everything from secret cargo shops to hot air balloons. amazon has consumer sales while exploring everything from drones to cell phones. And MSFT has windows and office and servers allowing it to explore all that MSFT research does. But indeed, MSFT is really well placed to continue to be a major player in the future and not just a niche provider for enterprise infrastructure. The minds of the young are nimble–while MSFT wasn’t cool for the 25+ crowd out there, the minds of 12+ are still open. So we will see…

        • NiXuqis

          You worry about cool, Nadella and I will worry about creating great enterprise services & software that leads to real profits. That is cool.

          • whatup12

            And you don’t think MSFT’s image hurts them as a consumer-oriented company? They already are well placed for enterprise–but indeed, they have a lot convincing the future generation that google docs, and google enterprise services are not sufficient. But thanks for the painfully superficial snarky comment…the irony was that it wasn’t cool at all.

          • NiXuqis

            Consumer loves companies that make great products, sold at the right time, at the right place, at the right price. Do that and your company will be considered cool.

            Let the schoolgirls and losers “worry” about being cool while those of us who produce quality and greatness has more important things on our minds and cool; an unintended by-product.

          • whatup12

            So Apple products are considered cool because they are sold at the right price as compared to highly intricate marketing strategies? Samsung Galaxy products didn’t require billions in obvious and less-than-obvious marketing strategies to sustain sales? And HTC didn’t just grab the Samsung Galaxy marketing person to take the HTC One and make it a hit? Indeed, the right product at the right price at the right time is not always enough. Stop oversimplifying what are actually quite complex market dynamics.

      • Bugbog

        I think you mean Ballmer, which Nadella is carrying forwards :)

        • NiXuqis

          No Ballmer was a paroxysm onto the company and hindered the hard work individuals was doing. With Nadella we at Microsoft now has a new CEO, a new strategy and a new optimism for the future. That has noting to do with Ballmer, except, his departure.

          • casaout

            Most of the stuff Nadella recently announced was done under the hands of Ballmer. You can’t do this stuff in a couple of weeks! Ballmer did a great job with Microsoft over so many years. He can be proud of what he achieved. Nadella now continues on this track and can now (slowly) start to move the company in the direction of his (and the other managers’) choice 😉 I understand that he has great charisma and is good for the “New One Microsoft”!

          • NiXuqis

            Ballmer did nothing but hurt a great company. It took awhile but the board finally saw that the profits the company was making could have been some with a monkey in charge while he continued to miss trend offer trend. The Board did the right thing and push him out. Nadella did in a few weeks things Ballmer refuse to.

          • whatup12

            Wow–you seriously think that things like office for ipad, new hardware management services etc came together in a few weeks? These things take months to operationalize…MSFT is a big ship and it takes a lot to turn it. Ballmer did more for future proofing that company than he will ever get credit for…but the issue was that he just wasn’t cool enough for you.

          • NiXuqis

            Office for iPad was completed in 2012 Ballmer did release it for he wanted to pursue hardware strategy and though holding Office would prop up sales for Windows. Nadella said if I am made CEO we will release IPad Office ASAP.

            Ballmer was afraid of the future, he wanted to live in the past where Windows had 90% market share of computer devices. Those days are gone and Nadella knows this and running the company Ballmer could not allow himself to do.

            I though the MS Board was the last to see how poor of a manager Ballmer was, I guess there is a few here that are even slower then the old MS Board…

          • whatup12

            Actually the people on this board tend to value evidence over opinion. And moreover, when they give an opinion, they tend to note it as such rather than as fact. The tech bloggers wet dream that office for ipad being ready in 2012 and held up by Ballmer has never been verified by anybody of substance. It is only reported by random bloggers who themselves apparently do not value evidence–and let’s face it, the new age of journalism does not value evidence as input given that they know that their audience will forget how their predictions were wrong anyways as they make their next ridiculous prediction. So indeed, while you must be privy to the notes from the board meetings giving you these really valuable insights…not. But indeed, we are slow and you are fast Ie, we value thoughtful inputs rather than fast assumptions. So superstar…what’s next for you?

  • J. L. Louis

    Maybe Microsoft should replace/integrate the Kinect’s speech recognition method (if it is local software/hardware based) this cloud based AI Technology.

    Also, in my opinion, it would be cool if Microsoft offers this AI technology as an API, or PaaS like Google is doing with its machine learning ‘Prediction API’, or IBM is doing (or planning to do) commercially with Watson. That way developers could use Cortana’s natural language processing abilities with their application, for a fee or for free.