Receive Our Tips
We offer custom made recommendation engines
- We help you to make recommendation decisions
- We provide our expertise in big data
- Offer the right things to the right people
Big Data Hits and Misses: Examples Found in Recommendation Engines
There are few tools more capable than recommendation engines when it comes to realizing a successful and satisfying advertising campaign. The basic structure is fail-proof in its simplicity: Take products and services on your site and display them to users that have already expressed interest in similar options.
The key to making appropriate recommendations is in gathering reliable and comprehensive data about the user in question. With the right set of data, you can make recommendations that they will like. The opposite is equally true—if your data set is incomplete or poorly managed, your engine will make ridiculous recommendations.
There are numerous blogs devoted to “recommendation engine fails,” and if you want your site to be taken seriously, it is a good idea to make sure that yours steers clear of them. Effective use of big data allows for the implementation of a very successful recommendation engine advertising campaign, but requires a dedicated development strategy.
Examples of Successful Recommendation Engines:
• eBay—One of the original adopters of the recommendation engine, eBay
might host one of the most successful examples of big data use in such a feature. eBay’s data centers perform billions of reads and writes each day, creating highly organized and scalable data structures that are used to automatically generate relevant recommendations based on user preferences.
In the past, eBay used a proprietary system for big data processing. This system was effective, but slower and more expensive than an outsourced solution. Now, eBay hosts more than 250 TB of data on external server clusters and operates a recommendation engine that captures and interprets customer preferences to make effective, useful recommendations every day.
• Pandora—Pandora’s online radio service could be the single most successful example of a big data-centric recommendation engine. The online radio service differs from any form of radio to come before in that it is entirely based on a recommendation engine, informed by its proprietary “music genome project” software that identifies individual musical elements that comprise taste.
Pandora is unique in that it uses a combination of human elements with automated ones to create an enormous data set, and then uses considerably powerful computing tools to make matches based on each listener’s choices on their site. The more a user listens to Pandora, the better its recommendations get as it forms a unique history for each member.
• Hulu—The popular video-streaming website uses content-based recommendation that is informed by massive amounts of user data. This data is put through an item-based collaborative filter to cut down on extraneous options that provide only quality recommendations, and is largely successful in doing so.
• Netflix—Modern Netflix (post-2009) makes successful use of big data in its content recommendations for users. The updated version of its proprietary recommendation software, Cinematch, uses a combination of 107 separate algorithms for determining user taste based on an enormous set of data gathered from every user who makes use of the site.
Recommendation Misses from Major Companies
While the use of big data in recommendation engines is always a necessary element of success, it does not guarantee success by itself. Many global brands and popular websites have earned the ire of their users through improper recommendation algorithms.
One of the more famous examples includes the earlier versions of Netflix’s Cinematch software, which had access to big data concerning its users, but improperly handled the data and created ridiculous suggestions such as the obscure and poorly-rated, “Shaolin Grandma” and “Norwegian Ninja” films for lovers of Bruce Lee-era kung fu movies.
Netflix overhauled its entire recommendation system in 2009 after holding a worldwide developer competition, installing a number of important filtering processes in its algorithm, and currently provides much better recommendations. No longer is the “Twilight” series mistakenly recommended next to “The Shining” to horror movie buffs.
Amazon’s recommendation system is often targeted for its capacity for jumping to conclusions about its users by over-analyzing its data set and characterizing users based on their purchases. The system makes extensive use of collaborative filtering, a process by which customer purchases are compared to the choices made by other customers sharing certain data elements.
This approach generally works successfully once it has a considerably large set of purchases to draw conclusions from. Beginning users of Amazon’s online store, however, tend to notice that the filtered results they are offered will draw strange conclusions—based not on their buying behavior, but on the buying behavior of people who bought the same, or similar, items
It is not uncommon for new users of Amazon to be recommended towards purchasing seemingly random collections of items: shovels, communist literature, and women’s razor blades as a result of purchasing a kitchenware set, for example.
Two Approaches for Using Big Data in a Recommendation System
There are generally two separate schools of application when it comes to managing big data in a recommendation system format. The best websites use a hybrid algorithm that makes use of conclusions drawn from both systems and then applies an additional filter at the end. These two approaches are characterized as follows:
• Collaborative Filtering—Conclusions arrived at through collaborative filtering are based on models of the behavior of previous users. This can be effective when users share similar traits, but it is important to maintain an effective means of interpreting your user data in order to arrive at the correct conclusions. Amazon is the key proponent of collaborative filtering.
• Content-Based Filtering—Recommendations made by content-based filters use the individual user’s historical information to inform choices displayed. Browsing history is a major component of this, and provides a reliable, if somewhat predictable, solution for finding quality recommendations. Hulu uses this approach most significantly.
Recommendation services have become a standard expectation of web users in a wide variety of domains. Implementing a successful one, however, is a nuanced process that merits a careful approach. When used correctly, recommendations improve business while making your users feel validated; improper use can alienate your audience. Use big data in your recommendation software with care!