Google Explains Why They Got Germany – France Match Prediction Wrong While Cortana Predicted It Right

Bing Football Experience

Last month, Bing team rolled out a new feature in which when a user search for “World Cup Predictions”, or any group matches (both preliminary as well as later in the single elimination rounds) they will display the chances of each respective team to win. This feature became quite popular as Cortana team integrated the same into Windows Phone. With the power of Bing, Cortana got all its predictions correct till now in this world cup. Google also joined the prediction arena with their own prediction for all the matches. Recently, Google got Germany – France match result wrong as Germany won over France making their prediction wrong. Meanwhile, Corana got it correct. Today, Google explained why they got it wrong in their blog post, read it below.

And so it went in the quarterfinals. We gave France a 69% chance, but we were wrong. Germany defeated France 1 to 0. That was the only game that upset our predictions. After going 8 for 8 in the Round of 16, we’re now at 11 for 12.

Why did we get Germany – France wrong?
World Cup teams are especially difficult to model because they play so few games together. USA coach Jurgen Klinsmann recently told the New York Times that he sees his players about as often as he sees his barber. If data is the lifeblood of a good model, we suffered for want of more information.

But, we know that in the same environment, others fared better in their predictions (h/t Cortana; their model relies more on what betting markets are saying, whereas ours is an inductive model derived from game-play data).

So, why did we get Germany – France wrong? In the first four games of the World Cup, France took more shots than Germany, had more shots on target, and their shots were from a more “dangerous location” (that is, closer to the goal). This information complements actual goals to form an ‘expected goals’ statistic in our model.

Moreover, in the first four games, Germany allowed their opponents to take more dangerous shots, and thus the expected goal statistic was higher for their opponents. And, it allowed their opponents to pass better in their third of the field. In the Germany-France game, France actually outshot Germany with 13 shots vs. 8 for Germany, and 9 vs. 6 on-target. With a little more luck on their side, they may have pulled ahead.

How Microsoft came up with the predictions?

The process of predicting outcomes of sporting events is unlike voting show predictions, as popularity and some sentiment signals do not play similar roles in determining who wins a contest.  Rather, the actions and performances of a handful of individuals solely determine who wins, loses, or ties.  For the tournament, our models evaluate the strength of each team through a variety of factors such as previous win/loss/tie record in qualification matches and other international competitions and margin of victory in these contests, adjusted for location since home field advantage is a known bias.  Further adjustments are made related to other factors which give one team advantages over another, such as home field (for Brazil) or proximity (South American teams), playing surface (hybrid grass), game-time weather conditions, and other such factors.  In addition, data obtained from prediction markets allows us to tune the win/lose/tie probabilities due to the ‘wisdom of the crowds’ phenomenon captured by the people wagering on the outcomes.

Both Google and Bing are not predicting that Germany will win over Brazil in the semi-finals. With the fact that Brazilian striker Neymar is out of the tournament, it is easy for anyone to come with those results. Anyway, let us see what happens!

Comments