Google Explains Why They Got Germany – France Match Prediction Wrong While Cortana Predicted It Right

27

Bing Football Experience

Last month, Bing team rolled out a new feature in which when a user search for “World Cup Predictions”, or any group matches (both preliminary as well as later in the single elimination rounds) they will display the chances of each respective team to win. This feature became quite popular as Cortana team integrated the same into Windows Phone. With the power of Bing, Cortana got all its predictions correct till now in this world cup. Google also joined the prediction arena with their own prediction for all the matches. Recently, Google got Germany – France match result wrong as Germany won over France making their prediction wrong. Meanwhile, Corana got it correct. Today, Google explained why they got it wrong in their blog post, read it below.

And so it went in the quarterfinals. We gave France a 69% chance, but we were wrong. Germany defeated France 1 to 0. That was the only game that upset our predictions. After going 8 for 8 in the Round of 16, we’re now at 11 for 12.

Why did we get Germany – France wrong?
World Cup teams are especially difficult to model because they play so few games together. USA coach Jurgen Klinsmann recently told the New York Times that he sees his players about as often as he sees his barber. If data is the lifeblood of a good model, we suffered for want of more information.

But, we know that in the same environment, others fared better in their predictions (h/t Cortana; their model relies more on what betting markets are saying, whereas ours is an inductive model derived from game-play data).

So, why did we get Germany – France wrong? In the first four games of the World Cup, France took more shots than Germany, had more shots on target, and their shots were from a more “dangerous location” (that is, closer to the goal). This information complements actual goals to form an ‘expected goals’ statistic in our model.

Moreover, in the first four games, Germany allowed their opponents to take more dangerous shots, and thus the expected goal statistic was higher for their opponents. And, it allowed their opponents to pass better in their third of the field. In the Germany-France game, France actually outshot Germany with 13 shots vs. 8 for Germany, and 9 vs. 6 on-target. With a little more luck on their side, they may have pulled ahead.

How Microsoft came up with the predictions?

The process of predicting outcomes of sporting events is unlike voting show predictions, as popularity and some sentiment signals do not play similar roles in determining who wins a contest.  Rather, the actions and performances of a handful of individuals solely determine who wins, loses, or ties.  For the tournament, our models evaluate the strength of each team through a variety of factors such as previous win/loss/tie record in qualification matches and other international competitions and margin of victory in these contests, adjusted for location since home field advantage is a known bias.  Further adjustments are made related to other factors which give one team advantages over another, such as home field (for Brazil) or proximity (South American teams), playing surface (hybrid grass), game-time weather conditions, and other such factors.  In addition, data obtained from prediction markets allows us to tune the win/lose/tie probabilities due to the ‘wisdom of the crowds’ phenomenon captured by the people wagering on the outcomes.

Both Google and Bing are not predicting that Germany will win over Brazil in the semi-finals. With the fact that Brazilian striker Neymar is out of the tournament, it is easy for anyone to come with those results. Anyway, let us see what happens!



About Author

Pradeep, a Computer Science & Engineering graduate.

  • Bugbog

    Waaah-wah-waaah! “Scwape me a river!”

  • Zicoz

    Googles way of doing this is just silly, you can’t consider shots made and shots on target unless you compensate for who the competition is.
    When there is a billion dollar industry that is built around finding out the chances for someone to win a game then I would look to that industry for guidance on how to predict matches and not look at some silly shot statistic.

    • Bugbog

      Probably ’cause the researchers doing the programming likely think it’s a sport comparable to Baseball, and it’s resultant use of statistics!?

  • jvs

    Google: Search engine, Bing: Decision Engine. Bing right, Google wrong.

  • CyberAngel

    Cortana is still Beta so it’s understandable that it can’t win always.

    • http://www.about.me/AngeloGopaul Eingoluq

      Dude you realize Cortana has been accurate 100% so far? Google has not. Your statement makes no sense. Did you read the article wrong thinking Google beat Cortana/Bing?

      • Alfred Soyemi

        I bet cyberangel read it wrong. My question or worry is how could anyone have read the article so terribly wrong?

        • CyberAngel

          Maybe it’s fake news?

      • CyberAngel

        No way can a game character in Beta using Bing beat an established Google Now with a better and bigger search engine! Something is wrong? Androids lesds the smart phone race and Nokia phone business was sold to Microsoft. The market share of the Windows Phone 8 is a few per cent. No cute female figure helps in that no matter if it luckily bets right a few games. Besides a halo ring on the phone like Lumia is not resembling that game Cortana AI figure. Now bring us a holographic naked blue lady and I don’t care if Monaco wins!

        • http://www.about.me/AngeloGopaul Eingoluq

          You are the definition of deluded. What does Androids market share have to do with the accuracy of prediction algorithms? Nothing. You are describing Google as though they are infallible.

          • CyberAngel

            We will see at the END GAME !!!

          • CyberAngel

            Netherlands 3:0 win over Brazil
            Cortana got it all wrong!
            Do you still defend Bing?

          • http://www.about.me/AngeloGopaul Eingoluq

            Yes…. why wouldn’t I? The chances either of them winning was almost 50 50. Anything could happen.

        • Anonoymous

          @CyberAngel:disqus I think you are a Microsoft protesters :P

          • CyberAngel

            Do you also protest now that Cortana made an error?
            Netherlands 3:0 win over Brazil

      • CyberAngel

        Anyway…Cortana blew it:
        Netherlands 3:0 win over Brazil
        What do you say now?

  • Rui Silva

    “Both Google and Bing are not predicting that Germany will win over Brazil in the semi-finals. With the fact that Brazilian striker Neymar is out of the tournament, it is easy for anyone to come with those results. Anyway, let us see what happens!”
    This doesn’t make sense. Bing is predicting that Germany will win… And if not, why Neymar out of competition says that it is that easy to say that Germany will not win?

    • Stuart

      I imagine it is a typo and means “now” instead of “not”

    • techieg

      Regardless of Neymar’s and Thiago Silva’s presence they cannot beat Germany. This Brazil team is not at all remotely spectacular, to say the least. Germany could do better as well but have shown themselves to be a much better team. I wager on Germany (buf of course soccer does not lend itself to logic or math, its the one sport I have played both in a team and outside of it, so I know).

      • RicardoDawkins

        This Brazil team sucks.. LOL. They are done against the German.

  • caywen

    Excuses.

  • Kickstar

    …”however, we are confident we know what you had for Breakfast this Morning, how many Selfies you took today, and how to resolve that Rash you spoke of in an Email this Morning. So who cares about Football”

    • http://www.currentlybreathing.net Ram Uppugunduri

      You got it totally right.

  • Andy

    So, did Google try to play off Bing’s data as simply being trends while they boast theirs as something more even though that’s incorrect?

    • Jean-Paul Accetta

      That’s what I got too. Pretty annoying how they downplay Bing pretending it’s really less competent or whatever. “They got it right, but it’s because they’re not as good.” Really?!

  • Rikkirik

    Google predictions are no better than betting markets it seems. Shame, shame, shame on Google. Bing predictions are much more refined and accurate. Enterprises and consumers should never trust Google data and predictions.

  • tomakali

    so, BEHIND… all google trends, analytics etc are all fake to make believe users that its right, but only beneficial for google… Google = Evil Data, Bing = intelligent data