I’m really deep in the rabbit hole of AB/Split/Multivariate testing right now. I’ve 50+ tabs open in Firefox and I’m soaking it all in–I’m not the only one who does that right? Like just start wildly opening every piece of information on a subject and reading everything like a fiend for three hours?
Anyway, I’m hot on the trail of something called the “Taguchi test” which was created by the gentleman on your right, Dr. Genichi Taguchi (Wikipedia). I have no idea what this is all about (I wasn’t a huge stats guy, although I did get an A- in my stat course at Fordham), but I feel like I’m getting somewhere. I just host getting somewhere doesn’t end up at Season Three of LOST (this guy does look the lab guy from Dharma movie film yes?).
- The Taguchi Method was developed 50 years ago and has been used with great success to optimize automobile and other product manufacturing. More recently, The Taguchi Method was applied to direct mail and web applications. The Taguchi Method takes a number of elements on a page with one or more alternatives for each element and dictates exact combinations that will allow you to estimate the positive or negative effect of each element/alternative.
- There are three extremely exciting aspects to this approach. First, by creating a “best page” using the best performing alternatives for each element, significant improvement can be achieved. Second, the length of the test cycle and the number of visitors required is surprisingly small. And finally, since the “recipes” are created using modular element/alternatives, using a solution like Offermatica, Taguchi tests can be designed and executed in a surprisingly small amount of time.
- Taguchi tests have been run on email, PPC ads and Landing Pages with great success. Where an AB Split Test might create a 5-10% improvement, a Taguchi test cycle will regularly return 25-45% improvement and has been known to improve results by 100% or more. A test cycle includes two weeks of testing a large number of elements in just two alternatives to identify which elements increase the likelihood of converting a visitor to a customer, a second test where the high-impact elements are tested with a greater number of alternatives, and a final test of the “best recipe” against the original page. The test cycle takes from a couple of days to a month depending on traffic and variance and can be designed and run without significant quantitative marketing or statistics experience.
And this company Offermatica has an excellent example written by Jamie Roche with lots of charts/images… very cool stuff!