For the next few weeks, my blog posts will primarily consist of re-authoring and re-building the Beginner's Guide to Search Engine Optimization, section by section. You can read more about this project here.

How Search Marketers Study & Learn How to Succeed in the Engines

The complicated algorithms of search engines may appear at first glance to be impenetrable, and the engines themselves provide little insight into how to achieve better results or garner more traffic. What little information on optimization and best practices that the engines themselves do provide is listed below:

The guidelines above aren't useless - some valuable tidbits are certainly present, including this from Yahoo!:

Many factors influence whether a particular web site appears in Web Search results and where it falls in the ranking. These factors can include:

  • The number of other sites linking to it
  • The content on the pages
  • The updates we make to our database
  • The testing of new product versions
  • The discovery of additional sites
  • Changes to the search algorithm—and other factors

Web Search lists results ranked by relevance and offers a combination of sites from the entire Web obtained by Yahoo! Search and from the Yahoo! Directory.

And this from Microsoft's Live:

  • In the visible page text, include words users might choose as search query terms to find the information on your site.
  • Limit all pages to a reasonable size. We recommend one topic per page. An HTML page with no pictures should be under 150 KB.
  • Make sure that each page is accessible by at least one static text link.
  • Don't put the text that you want indexed inside images. For example, if you want your company name or address to be indexed, make sure it is not displayed inside a company logo.

    • These tiny snippets are, thankfully, just the tip of the iceberg. Over the 12+ years that web search has existed online, search marketers have found methodologies to extract information about how the search engines rank pages and use that data to help their sites and their clients achieve better positioning. Surprisingly, the engines do support many of these efforts, though the public visibility is frequently low. Conferences on search marketing, such as the Search Marketing Expo, WebMasterWorld & Search Engine Strategies series attract engineers and representatives from all of the major engines. Search representatives also assist webmasters by ocassionally participating online in blogs, forums & groups (these are listed below in Appendix B).

    However, there is perhaps no greater tool available to webmasters researching the activities of the engines than the freedom to use the search engines to perform experiments, test theories and form opinions. It is through this iterative, sometimes painstaking process, that a considerable amount of knowledge about the functions of the engines has been gleaned. A common process for testing might look something like this:

    1. Register a new website with nonsense keywords (e.g. ishkabibbell.com)
    2. Create multiple pages on that website, all targeting a similarly ludicrous term (e.g. yoogewgally)
    3. Test the use of different placement of text, formatting, use of keywords, link structures, etc. by making the pages as uniform as possible with only a singular difference
    4. Point links at the domain from indexed, well-spidered pages on other domains
    5. Record the search engines' activities and the rankings of the pages
    6. Make small alterations to the identically targeting pages to determine what factors might push a result up or down against its peers
    7. Record any results that appear to be effective and re-test on other domains or with other terms - if several tests consistently return the same results, chances are you've discovered a pattern that is used by the search engines.

    Here's an example of a test that we at SEOmoz have performed in the past:

    Test Step 1

    In this test, we started with the hypothesis that a link higher up in a page's code would carry more weight than a page lower down in the code. We tested this by creating a nonsense domain linking out to three pages, all carrying the same nonsense word exactly once. After the engines spidered the pages, we found that the page linked to from the highest link on the home page ranked first and continued our iterations of testing.

    Test Step 2

    We had some concerns that the text inside the link might be the source of the rankings, and so changed the link text on the homepage to nonsense characters. Still, the pages ranked in order of highest to lowest rank. Over several more iterations (changing up the linked-to pages, changing the terms, mixing up term usage, etc.) and repeat testing, we found the pattern emerged again and again, and decided to call it "confirmed." It appears that the three major engines (Ask never indexed the domain) all place some amount of higher weight on links higher in code than lower down. Obviously, this is a time consuming and tedious process, but it can help search marketers to understand the basic patterns of how search engines determine rankings.

    This process is certainly not alone in helping to educate search marketers. Competitive intelligence about signals the engines might use and how they might order results is also available through patent applications made by the major engines to the United States Patent Office. Perhaps the most famous among these is the system that spawned Google's genesis in the Stanford dormitories during the late 1990's - PageRank - documented as Patent #6285999 - Method for node ranking in a linked database. The original paper on the subject - Anatomy of a Large-Scale Hypertextual Web Search Engine - has also been the subject of considerable study and edification. To those whose comfort level with complex mathematics falls short, never fear. Although the actual equations can be academically interesting, complete understanding evades many of the most talented and successful search marketers - remedial calculus isn't required to practice search engine optimization.

    Through methods like patent analysis, experiments, and live testing and tweaking, search marketers as a community have come to understand many of the basic operations of search engines and the critical components of creating websites and pages that garner high rankings and significant traffic. The rest of this guide is devoted to explaining these practices clearly and concisely.

    ... ooph. This is tough stuff - as always, please provide your input and edits. We'll be making changes to all of this content before it goes into the finished guide (and Mystery Guest even offered to edit - yay!)

    Today's web roundup:

    I'm on David Brown (aka NEOSEO)'s podcast show tomorrow morning at 10am and it's already 2am... Man, I never get to sleep...