If you’re reading this article, you’re either intrigued or already seriously looking to find the best Recommendation Engine technology that offers predictive personalization for your email marketing program. To properly evaluate this kind of technology for your business, below you’ll find a list of questions to ask as well as more detailed criteria you should keep in mind.
At the end of this post the list of questions has been condensed to serve as a quick reference guide when you're in the middle of vendor negotiations, however you might want to review a bit more detail in advance. Below we've expanded on each topic in detail to facilitate your preparation.
What data is used to generate the content recommendations?
LOOK FOR:
- Engines that use both explicit and implicit data of website visitor behavior. For example, information that is provided intentionally via a purchase or download as well as dwell time and pages browsed. Avoid Engines with data inputs that are too limited, which may result in over-weighted bias towards popular or trending items.
- Also look for Engines that get running quite fast, or have the ability to supplement the model with starting data to avoid the “cold start problem.” Because these models must capture and analyze behavior before they can start producing good recommendations, it might take time to become smart enough. Depending on the sophistication of the algorithms and the volume of your website traffic, the Engine could take several weeks to become productive. The ability to upload history gives the system a jumpstart.
- Another important data topic is previous browsing behavior that occurred before a transaction took place, because consumers will often browse several times before taking that first explicit step. That earlier behavior is important for the model to understand broader categories of interest and ensure those next best recommendations are well-informed. Irrelevant recommendations will cause consumers to lose trust in your brand.
- Finally, supplementing your first-party data with third-party data may also be useful in some industries, so be sure to discover whether that applies to your business case.
Does the data model use predictive analytics and machine learning to produce the recommendations?
LOOK FOR:
- The Best Recommendation Engines use multiple algorithmic methods such as collaborative filtering and content-based filtering.
- Reports, case studies or data should support an improvement in performance after technology implementation, as well as demonstrate a continuing trend of self-optimization over time.
- The system should pay attention to how subscribers interact with the email recommendations and feed that data back into the model. In most cases the faster this loop is closed the better.
Does the Recommendation Engine work in real-time?
LOOK FOR:
- Technologies that are running continuously, communicating instantly, and work in real time from end-to-end.
- New behaviors of a customer on a website should instantly inform the predictive data model, which should immediately update the content recommended. It’s worth noting that even if an email campaign has already been sent, the latest systems can push the updated content into each email, right inside each inbox.
- Watch out for technologies that use batch processing to update recommendations on a less frequent basis such as daily or weekly, as well as systems that have recommendations frozen at send time – these may not be ideal for your business needs and expectations.
Is the technology easy to integrate?
LOOK FOR:
- Testimonials, satisfaction surveys and references should be available that demonstrate ease of integration and deployment.
- You’ll want a transparent overview of the costs involved beyond price. One of the biggest problems of Recommendation Engines today is that they are embroiled in complex systems that are expensive to buy and require teams or agencies to run. You may even have to replace your entire marketing automation platform to obtain this capability, and then pay the vendor extra for technical service support.
- If the workload and cost seem too heavy don’t be discouraged – there are also new advances in technology that avoid complex integrations and can work with your existing email platform.
Is the interface application easy to use?
LOOK FOR:
- Applications that you can demo as well as testimonials from customers about what it takes to use the technology should help you assess the time to create one personalized campaign and confirm that it works.
- AVOID complexity. Studies show that despite the oodles of bells and whistles in some Recommendation Engines, the majority of email marketers aren’t using them. Furthermore, when given the opportunity to adjust recommendations presented to them, subscribers don’t engage. These manual fine-tuning features might seem important, but in reality they are for edge cases if anything at all, and their value has not been proven. The best approach is to go with something simpler that’s optimized for the 90% and then rely on machine learning to fine-tune the engine.
Is the Recommendation Engine proven in your vertical?
LOOK FOR:
- Case studies should be available for your vertical demonstrating a positive impact on the traditional email metrics, as well as return-on-investment (ROI) versus cost. Aggregate data by vertical is also useful to evaluate. If there are no case studies or testimonials relating to your type of business, you should probably walk away, unless you have agreed to be in a beta or pilot of some kind.
- Be sure to confirm how much data the model was “trained on.” Low numbers of profiles mean the model has not had to deal with enough long-tail cases. Several million subscribers or consumer profiles is a solid basis for a more intelligent data model.
