Team “BellKor’s Pragmatic Chaos” Wins Contest; Another $1 Million Challenge is Announced
After almost three years and submissions by more than 40,000 teams from 186 countries, Netflix, Inc., the world’s largest online movie rental service, today awarded the $1 million Netflix Prize to a team of engineers, statisticians and researchers who achieved the competition’s goal of a 10 percent improvement over the accuracy of the Netflix movie recommendation system when the competition was launched in Oct. 2006. Netflix members already are benefiting from improvements Netflix Prize contestants have contributed to the recommendations system.
Moments after bestowing the $1 million prize, Netflix announced a second $1 million challenge, asking the world’s computer science and machine learning communities to keep the improvements coming.
The team “BellKor’s Pragmatic Chaos,” the merging of three teams that had previously competed against one another in the contest, received the $1 million Netflix Prize in an award ceremony hosted here today by Netflix Co-Founder and CEO Reed Hastings and Chief Product Officer Neil Hunt.
“We had a bona fide race right to the very end,” said Mr. Hastings. “Teams that had previously battled it out independently joined forces to surpass the 10 percent barrier. New submissions arrived fast and furious in the closing hours and the competition had more twists and turns than ‘The Crying Game,’ ‘The Usual Suspects’ and all the ‘Bourne’ movies wrapped into one.”
The winning team is comprised of software and electrical engineers, statisticians and machine learning researchers from Austria, Canada, Israel and the United States. All seven team members – Bob Bell, Martin Chabbert, Michael Jahrer, Yehuda Koren, Martin Piotte, Andreas Toscher and Chris Volinsky – attended the awards ceremony. It was the first time all seven had met one another in person. How the $1 million is split is to be determined by the team.
Netflix said “BellKor’s Pragmatic Chaos” edged out a team called “The Ensemble,” another collaboration of former competitors, with the winning submission coming just 24 minutes before the conclusion of the nearly three-year-long contest. The competition was so close and the submissions so sophisticated that it took a team of external and internal judges several weeks to validate the winner after the contest closed on July 26. The Netflix Prize external judges are University of California, San Diego Professor Charles Elkan and University of California, Irvine Professor Padhraic Smyth. The internal judges are Netflix senior engineers Stanley Lanning and Jon Sanders.
The contest’s rules require the winning team to publish its methods so that businesses in many fields can benefit from the work done. The winning submission and the previously hidden ratings used to score the contest will be published at the University of California Irvine Machine Learning Repository. The team licensed its work to Netflix and is free to license it to other companies.
When Netflix launched the Netflix Prize in October 2006, it made available to contestants 100 million anonymous movie ratings ranging from one to five stars, the largest such data set ever released. All personal information identifying individual Netflix members was removed from the prize data, which contained only movie titles, star ratings and dates but no text reviews. The challenge was to improve upon the company’s ability to accurately predict Netflix members’ movie tastes by 10 percent, a hurdle Netflix scientists were not able to overcome on their own over the last decade.
“Accurately predicting the movies Netflix members will love is a key component of our service,” said Dr. Hunt. “This extreme level of personalization is like entering a video store with 100,000 titles and having those that are most interesting to you fly off the shelves and line up in front of you. We take the guess work out of renting by presenting the movies and TV episodes we believe each Netflix member will most enjoy,” he added.
Netflix Prize 2 – The Next $1 Million Challenge
While the first Netflix Prize solved the tough challenge of accurately predicting movie enjoyment by Netflix members who have provided ratings on an average of 50 or more other movies, Netflix Prize 2 focuses on the much harder problem of predicting movie enjoyment by members who don’t rate movies often, or at all, by taking advantage of demographic and behavioral data carrying implicit signals about the individuals’ taste profiles. As with the first Netflix Prize, the sequel will also be an open competition with winning teams owning their solution to license to Netflix and other companies. Success in this problem will enable businesses to deliver superior service to new customers much sooner in their lifecycle, without requiring or waiting for the customer to provide the rich data points that underpinned the first Netflix Prize.
The new data set, providing more than 100 million data points, will include, among other things, information about renters’ ages, genders, ZIP codes, genre ratings and previously chosen movies. As with the first Netflix Prize, all data provided is anonymous and cannot be associated with a specific Netflix member.
Unlike the first challenge, this contest has no specific accuracy target. In fact, Netflix said today that the company and the judges have little idea how far the world’s foremost experts can push this data to derive useful predictions. Instead, $500,000 will be awarded to the team judged to be leading after six months and an additional $500,000 will be given to the team in the lead at the 18-month mark, when the contest is wrapped up. Once again, Netflix will require the winning team to publish its methods.
The Netflix recommendation engine spans the 100,000 DVD titles in the Netflix catalog and is an essential element of the company’s movie subscription service. Each of the 10.6 million Netflix members enjoys a personalized member Web site that enables them to rate movies on a one to five star scale. Netflix combines those individual ratings into a database of more than three billion movie ratings and employs proprietary algorithms and software to identify movies that tend to be rated highly (or poorly) by people with similar tastes. Netflix has already enhanced these algorithms using innovations from the winners of two annual Netflix Progress Prize awards. The accuracy of this software has been praised by movie critics and members alike and enables Netflix to fulfill its goal of connecting people with movies they’ll love.
Complete details about the Netflix Prize are available at www.netflixprize.com.
About Netflix, Inc.
Netflix, Inc. (NASDAQ: NFLX) is the world’s largest online movie rental service, with more than 10 million subscribers. For only $8.99 a month, Netflix members can instantly watch unlimited movies and TV episodes streamed to their TVs and computers and can receive unlimited DVDs delivered quickly to their homes. There are never any due dates or late fees. Netflix members can exchange DVDs as often as they want using a postage-paid return envelope. Members can choose from a vast selection of DVD titles and a growing library of movies and TV episodes that can be watched instantly. Netflix is partnering with leaders in consumer electronics to bring to market a range of devices that can instantly stream movies and TV episodes from Netflix directly to members’ TVs. These devices currently include Blu-ray disc players and new Internet TVs from LG Electronics; Blu-ray disc players from Samsung; the Roku digital video player; Microsoft’s Xbox 360 game console; TiVo digital video recorders; and, soon, Internet TVs from Sony and VIZIO.
