Multivariate Content Testing: Where Guessing Games Go to Die

q1-2010-Multivariate-Content-Testing-Where-Guessing-Games-Go-to-Die


Once visitors reach a landing or conversion page, the content that they encounter is the overriding factor that will determine what they do.


While a good creative team or marketing group can come up with a number of variations for developing powerful copy, engaging images and compelling messages, selecting the right elements for the target audience is often a challenge. Without an analytical approach to guide these decisions, they are often made based on personal preference or the persuasiveness of a vocal team member.


Enter multivariate content testing, a powerful tool for selecting the right content elements in order to maximize the desired actions from the site’s visitors. While multivariate content testing sounds complicated, the concept is really pretty simple. It is, basically, a methodology for selecting and presenting variations of site content while measuring the results of each variation based on key factors, such as page conversion rate (e.g., purchasing an item, signing up for a newsletter, etc.).


Multivariate testing goes beyond traditional A/B split testing to optimize site content in a much more efficient and complete manner. One of multivariate testing’s underpinning principles is that none of the content areas are independent of the others in their overall effect on the visitor. By recognizing this, multivariate testing can prevent brand managers from making a drastic and damaging change to their websites even though A/B split testing could have potentially made this move seem like a good idea.


Here is an example scenario to highlight the difference between these two methods:

  • With A/B split testing we see that “Image A” performs better than “Image B,” so we eliminate “Image B” from that page.
  • With Multivariate Testing we see that “Image B” actually performs better than “Image A” when it is combined with “Paragraph B.” We now have the best performing combination because we were able to learn that “Image B” just needed to be placed in the right context for users.


The differences might be subtle, but sometimes a single percent change in conversion can make all the difference.