Yesterday's announcement of the SEOmoz premium landing page contest created an exceptional buzz - we've already had more than a dozen people contact us about creating their version of the document. In addition, I received some excellent advice from two experts in the field - Jon Mendez of OTTO Digital and Bryan Eisenberg of FutureNow. To help anyone and everyone who's in the field of landing page design & creation, I though it would be valuable to share their wisdom.

Jon wrote about our contest on his blog. He notes some of the big issues with multivariate testing:

...Traffic and associated conversions will help determine the first part of test design, what kind of test to create. A good rule of thumb is 100 conversions per page (or what refer to as recipe) for any test. So an A/B/C test should need about 300 conversions to reach confidence and stability and a 4x3 Multivariate test being an L9 array with 9 recipes would need 900 conversions. WARNING: This is just a rule of thumb. Your results may vary!...

...One of the biggest challenges we face at OTTO Digital is running tests for clients around set timeframes like seasonality and promotions. Anytime your results need to be achieved by a certain date you can compromise confidence because results are unpredictable. Ideally the data and most importantly the stability of the data will determine when your test is over, not your marketing calendar. Test stability is best determined by analyzing results over a period of time and looking for performance fluctuations. Stability is achieved when the tested elements exhibit similar performance behavior across enough data...

...With what Rand has disclosed about his traffic and conversion rates I would suggest running 5 page variations against each other….an A/B/C/D/E test. These pages should be selected based on not only how they look, but how they look compared to one another. This test design should produce a nice data set with high confidence in the 3-4 weeks timeline Rand has suggested.

Another option would be starting with a larger test, maybe 10 pages, and then look to drop pages from the test if they show consistent poor performance, thus increasing the distribution of traffic across the remaining pages and still working within a feasible timeframe for results. The downside to this scenario is that it only works if there is clear confidence among a handful of better performing pages.

Jon's giving truly solid advice here, not just for the SEOmoz test, but for any kind of testing that's performed. He's also done an excellent job of convincing me that we'll need to continue to lean on his expertise throughout the conversion testing process - conveniently, his number's now tattooeed on Rebecca's forehead so I can look at her whenever I need to call :)

Bryan Eisenberg's advice centered on something that may be even more interesting to those actually participating in the contest. Since his insight came over the phone, I'll attempt to list it in the bullet points below:

• We should continue to offer three different versions of pricing - monthly vs. 6 months vs. one year - and list them horizontally in a table showing the savings side by side. Apparently his experience suggests that this design would be very effective.
• Differentiation is critical to making the landing page successful - Bryan felt that answering the question "why should I spend my time in premium content vs. all the other SEO resources, blogs, forums, etc?" is key to the conversion process.
• The laundry list of features needs to become a set of benefits - this parallels what Roberta noted in her review of the page.
• Bryan also offered up links to several of Grokdotcom's best posts on the subject of landing page design:
  • The Value of First Impressions
  • Push Your Customers' Buttons
  • Hitting the Landing Page Optimization Wall
  • Create Landing Pages that Convert
• The Q+A "consulting" value isn't coming through in the current page and is likely to have a big impact on conversion rates.

In addition to Bryan & Jon's help, I've also been asked to reveal some critical information about SEOmoz's stats since the launch of premium content so contestants can craft better entry pages. Here you go!

Data is from February 1, 2007 - July 17, 2007:

• We've had 58,669 visits to the premium landing page total, resulting in 374 individual purchases of articles and 1498 premium signups
• Visitors reached the premium landing page in the following ways:
  • Banners (the overlay banner and the inline blog banner - launched in mid-June) - 2,455
  • Paid Search (Google AdWords campaign - launched in late June) - 582
  • Email Campaign (Rand's email signature - just launched) - 18
  • Organic Search (this isn't direct, just visitors who came through organic search, then got to the premium landing page) - 8,084
  • Link Referrals (as above - not necessarily direct) - 19,944
  • Direct Access (from feed reader, type-in traffic, bookmarks, etc) - 27,593
• The top referrals that bring traffic to the premium page are:
  • #1 - The Blog
  • #2 - The Home Page
  • #3 - The Page Strength Tool
  • #4 - The Search Ranking Factors Article
  • #5 - The Crawl Test Tool
• The top direct, external link referrers to the page came from:
  • SEO on a budget from SEOish
  • Roberta's Review on Copyblogger
  • SEM Scholarship on Marketing Pilgrim
• The top direct search phrases (non-ppc) that brought traffic were:
  • premium membership
  • premium member
  • premium content
  • seomoz premium
  • seo membership
  • premium members
  • seomoz premium membership
• The terms we currently bid on in PPC are:
  • SEO Consulting
  • Search Engine Optimization Consulting
  • SEO Tools
  • Search Engine Optimization Tools
  • SEO Training
  • Search Engine Optimization Training
  • SEOmoz
• The vast majority (85%+) of conversions have been to visitors who used the inline ads or followed a link from the blog or an article on the site. This serves as further evidence that premium membership appeals very well to those familiar with the brand, but poorly to newcomers.

Hopefully that data will provide enough relevant information for landing page creators to have a fair idea of what to aim towards. I think the big takeaway is - don't optimize for search phrases, optimize to convince those with little - moderate comfort with the brand to take the leap. At least, that's how I'd do it if I entered the contest. 

In related news, we received our first submission to the landing page competition, courtesy of Mystery Guest. Be warned, it's semi-NSFW:

 
Larger Version

I'm not entirely convinced that we can display the page, despite it's comic genius. As Mystery Guest has shown in the past, she can be pretty mean to Neil. Still, it's a valiant effort, and you can see why she's quite clearly the best thing to ever happen to me, period (hence all this business). :)

BTW - Tomorrow we launch the new SEOmoz pricing (sometime in the mid-morning) and we'll also be announcing two new premium member services. Plus - more great content for free members is also on its way.

p.s. Nearly forgot to mention that Offermatica has now added to the prize pot with a 3-month trial subscription of the basic version of their software (worth $30,000 - wow).

p.p.s. Yes - I know there's been a lot of "self-promotional" type blog entries in the past few weeks; with the exception of one more tomorrow, we'll get right back to the business of blogging useful SEO tips & advice without the SEOmoz-centric add-ons.