As I mentioned in a previous post about the numerous reasons for organic search engine ranking discrepancies, there are a wide variety of tools that can help simplify the process of checking a site’s rankings for multiple keywords across the primary search engines. In this post I will explore four methods for tracking rankings and compare the features, costs, and results of each.

Methodology

I tested the same set of 78 keywords on Google, Yahoo!, and MSN using each of the four methods listed below. The destination domain was AmericanAqua.com, a Michigan-based small business that provides a variety of services including bottled water delivery, salt delivery, water treatment, and water analysis. The keywords selected were a mix of branded terms, product/service terms, and geographic terms. As a control, several broad, generic keywords were chosen where the company has no supporting content on their site and thus would not expect to be ranking. Only the first listing for the destination domain was counted, so if the site was listed in positions 4 and 12 for the same keyword on the same engine, only the position #4 ranking was counted in the results.

In order to control for as many variables as possible, the data was collected in a 4-hour window from the same computer, using the same browser. I was not logged in or cookied by any of the engines at the beginning of the data collection period. The automated rank checkers were set to query no faster than once every 5 seconds to avoid triggering the engines’ anti-scraping mechanisms. Only the first 30 organic results (3 pages) were scanned because let’s face it, if your site doesn’t appear in the first 3 pages you might as well be invisible.

The Contenders

• Web Position 4 is a feature-rich software package that was launched by WebTrends in 2005. Judging by the age of the content and “latest” press releases on their site, there hasn’t been much activity or improvement of the software since then. The software is available in “Standard” and “Professional” packages. Only results from the Professional version are included below.
• SEOBook’s Rank Checker Firefox Extension is a free product offered by Aaron Wall. Once downloaded, the tool itself lives within your Firefox browser as an extension and does not require any additional software.
• SEOMoz’s Rank Checker is another web-based tool offered by the SEOMoz.org team. Unregistered users can run a limited number of queries per day for free, but premium members get unlimited usage for a monthly fee.
• Manual Rank Checking is done the old fashioned way…with a keyboard, mouse, and a spreadsheet. No bells and whistles, just you and a mild case of Carpal Tunnel Syndrome.

¹Web Position 4 Standard edition is $189 and performs all of the same organic rank checking functions. The Professional edition has a few useless add-ons that are not useful for experienced SEO’s or no longer considered useful, i.e. search engine submissions.
²SEOMoz allows registered, non-Premium members to check up to 5 keywords a day for free. Unlimited usage of their Rank Checker tool requires a $49 per month Premium membership, although discounts are given for 6 and 12-month subscriptions.
³For some reason there is an arbitrary 45 minute delay in running Web Position reports on Yahoo after about 50 keywords. Total running time was 84 minutes, but 45 of those were just waiting around.
⁴This probably would have taken much longer, but I didn’t bother checking the ranks of about 150 keywords (50 KWs * 3 engines) that did not show up in any of the other tools.

The Results

Despite the various data collection methods and reporting tools, all 4 data collection methods returned the exact same rankings nearly 72% of the time. Where they were off, the average standard deviation was only 0.84, meaning the difference between collection methods was off by less than 1 position on average. That indicates that the tools are consistently pulling very similar rankings.

However, it should be noted that of the 78 potential rankings on each engine, at most we saw 29 keywords ranking at one time. When the non-ranking keywords are removed from the previous calculations, we only see the same results from each tool 35% of the time with a much higher standard deviation of 1.95.

The complete set of data and calculations are stored in this Google Spreadsheet for you to examine.

Interesting Findings

The following tidbits stuck out while collecting and analyzing the data:

• The four rank checkers agreed most often on MSN rankings. Of the keywords that ranked, the standard deviation on MSN was only 1.08, compared to 1.99 on Google and 2.77 on Yahoo.
• Yahoo’s results were the most inconsistently reported by the four rank checkers.
• SEOBook’s Firefox Rank Checker was the fastest, but missed a few rankings than the other tools found.
• SEOMoz’s tool only allows one query at a time, which slows down the data collection considerably for larger keyword sets.
• SEOBook’s Firefox Rank Checker managed to delete one word from the file export, which caused a few minutes of working backwards to determine what was dropped. I ran that one again and it was fine.
• SEOMoz’s tool reports results as “Found on page X, position #Y”, which slows down the process even more to manually convert to an exact position out of 30.
• Many of Web Position’s additional features are no longer selling points and are practically worthless, such as their search engine submission tool. The only compelling features are the ability to run multiple keywords across many engines and the data trending and export functions.

Conclusions

Given the similarities in results between all four methods, I recommend SEOBook’s Rank Checker Firefox Extension because it is free, fast, can handle multiple keywords across the top engines, and has sufficient data export capabilities. It won’t trend your results over time, but a simple Excel spreadsheet or database can make some great charts and graphs from the raw data. Each rank checking method has their own pros and cons, but Aaron’s tool has the right combination of cost (free) and features (just enough to get the job done, without too many bells and whistles). Other tools have more robust feature sets but are too expensive, or are free but too time-consuming and limited in functionality.

None of the organic search ranking tracking methods proved to be dramatically more or less accurate than the others. There are many external variables that could influence rankings by a few positions or more, and these are likely the main cause for the discrepancies between the various tools.

These data only represent rankings at one point in time. The best way to determine the effectiveness of your SEO campaigns is to measure rankings across multiple engines at regular intervals so you can examine the trends. Do not put too much emphasis on any one data point, since they are very likely to fluctuate over the short term. Instead, pay attention to the longer-term trends to look for signs of improvement in your rankings.

As with any software decision, you must draw your own conclusions based on what is important to you. What features do you think are lacking from today’s rank checkers? How should they be extended to be more useful? Which do you use, and which are your favorites?

One of the challenges of any SEO’s job is to accurately report the results of his or her work. If you can measure it, you can improve it, right? Traffic, site usage, and online conversion metrics are easy. My 15 year old cousin with Google Analytics can handle that before breakfast and still have time left over to sign in to Facebook and play his bingo on Scrabulous (I still win…usually).

What about organic search rankings? Those are a little harder to nail down. But as SEO’s, isn’t that what we you contracted to improve? How can you show off your SEO prowess without a basic rankings report? Consider the increasing number of variables that are factored into search engine results pages (SERPs) and the frequency of algorithm updates and you might begin to understand why my results might look different than yours for the same keyword on the same engine. Just off the top of my head, my results could be different than yours due to one or more of these reasons:

• Geographic differences in our IP addresses
• Whether or not one of us is logged in to a search engine account
• Different search histories
• Search preferences such as # of results per page, profanity filters, etc
• We hit different data centers (server farms) that are in different stages up nearly constant updates
• Random SERP and algorithm testing conducted by the engines

As if that wasn’t enough, reporting a ranking for one keyword on one engine at one particular point in time is misleading because it represents just one data point without much context. Who knows if that particular ranking is better or worse than an average ranking? What we really like to see are trends and rolling averages. It’s the only clear way to assess an SEO campaign’s impact on rankings over time.

Automated Rank Checker Basics

So, how does a smart SEO repeatedly track rankings for multiple keywords across many different search engines without manually searching, counting, and storing the data in a spreadsheet? I prefer to use automated tracking software. There are several different products on the market, but they all basically do the same thing: simulate a search and parse the source code of the resulting SERP to look for your site’s rank. This is usually called “scraping” or “data mining” and is typically frowned upon or blocked by the search engines because it consumes server capacity in much greater quantities than a human would. Consider that a human usually spends several seconds scanning a single SERP and analyzing the results. A scraper can search several pages per second, which significantly adds to the search engine’s server load. Enough scrapers running at the same time could slow the server to a crawl and degrade human users’ experience.

A smart SEO should be responsible and set their scraper to crawl at more reasonable rates (1 request every 3-10 seconds) and from different IP addresses if possible. Proxy servers work well for this if you have access to them. This will lessen the load on the search engine and reduce the risk of your IP address being flagged and potentially banned from the engine for a while.

Interpreting the Results

Once you have selected a rank checker, run the same keywords on the same engines at least once a month. Plot your keyword rankings (y-axis) against time (x-axis) on a graph if your program doesn’t do this for you. Do you see rankings generally improving over time? Congratulations. Your SEO efforts might be doing the trick.

If your rankings are flatlined or headed downhill, re-evaluate your strategies and tactics. Are your keywords too competitive? Are you focusing too much on on-site or external factors? Are your tactics in line with the engines’ guidelines? Most of all, give it time. Depending on how frequently your site is crawled, updates or new links can take weeks or months to impact your rankings.

Lastly, check your site analytics and referral logs to see if there is a corresponding fluctuation in traffic from your targeted keywords. It may be helpful to overlay your SEO tactics on the x-axis to look for patterns. Remember, correlation doesn’t necessarily indicate causality, but if you see repeated instances of rankings changing after you make changes to the site, it might be a good indicator that you are doing something right. There might also be some lag time as previously mentioned.

Getting the Client on Board

Educating a client to understand that rankings can (and frequently do) change overnight is just one of the things that makes SEO such a hard sell sometimes. Just like other metrics, it is wise to review rankings data as a trendline rather than fixed data points. Compare the data to last week, last month, or last year to look for improvements. Overlay your SEO campaign history on the graph to isolate the impact of your tactics.

Most of all, learn from your past successes and failures. Learning what DOESN’T work is just as important as learning what DOES work.

I highly recommend this article from page A1 of today’s Wall Street Journal for anybody interested in why Microsoft’s (NASDAQ:MSFT) buyout offer for Yahoo (NASDAQ:YHOO) fell through at the last minute. It reads less like a business story and more like a suspense novel.

It has all the intrigue of a Hollywood blockbuster: two forces uniting against a common enemy, secret meetings at remote aiports, a woman with a red umbrella to signal that the coast is clear, private CEO-to-CEO meetings in the back of a private jet, and a promise of a sequel!

Best of all, it continues to promote and elevate the prominence of the search engine industry and all of the secondary industries built around it. The drama may cause more than a few sleepless nights for Steve Ballmer, Jerry Yang and their armies of foot soldiers, but it is first class entertainment for those of us in the peanut gallery.

And if you prefer dialogue to suspense, it doesn’t get much better than one Microsoftie’s face-saving quote as the closing credits roll,

“They believed that we needed them much more than they needed us. Ultimately we called their bluff.”

Get ready for the sequel where Microsoft returns to buy just the search part of Yahoo’s business. This time it’s personal!

As reported on TechCrunch and ZDNet, Adobe has created a standardized format for its Flash development suite that will allow search engines to index dynamically generated content, which was not previously possible. What does this mean for the industry? Frankly, it means that “existing and future” Flash sites will be more accessible to search engine crawlers which will result in many previously invisible websites being indexed and ranked in search results. Google is already incorporating the new capabilities into their search platform, and Yahoo! reportedly still “has some work to do.”

This is good news for designers that prefer Flash for its flexibility and animation capabilities. However, it also opens up a whole new series of questions that will have to be explored and tested by the SEO community:

• Will this newly indexable content start out with zero site history? As we know, the age of a domain plays a role in calculating its authority and credibility. If a Flash site has been around for years but is just now becoming “visible” to a crawler, can its age be accurately determined?
• Without a hierarchical markup system, how will crawlers treat text and images of varying importance? For example, in HTML we can use the <h1> and <strong> tags to emphasize particular pieces of content. What is the Flash equivalent, and will those tactics be more or less impactful than their HTML counterparts?
• How will crawlers treat Flash sites that have a properly optimized HTML framework? As an SEO, I know how to create a crawler-friendly HTML version of Flash content to aid with crawler accessibility. Will Google and Yahoo treat sites with both formats differently or prefer one over the other?
• Will sites built entire of Flash be treated differently than HTML sites with Flash components? Good SEO and user experience (accessibility) dictate that Flash should not be used to create an entire site, but rather certain interactive elements within it. Will two sites with similar content be treated differently if one is built entirely in Flash and the other is a Flash element wrapped in standard HTML?
• How will the use of Flash adjust to incorporate deep links? Most purely Flash sites have no unique URLs for each “page” because all of the content exists within one .swf file. Therefore, most inbound links would have to point to the top-level URL, regardless of where the desired content resides within the rich media application. This will help the domain-level rankings but not individual pages.
• Will Flash designers adapt their use of analytics to properly measure the traffic and conversions from newfound organic search traffic? There is a little extra work required to properly track Flash site usage statistics using traditional site analytics packages. I’m guessing relatively few Flash sites have this capability built in, and even fewer are going to retrofit their sites with action tags.
• How will non-text digital assets (images, video, audio, etc) be optimized within Flash? Will the same rules apply, and if so, will digital assets embedded in Flash be more, less, or equally weighted compared to their HTML counterparts?

I’m sure this list of questions will grow over time and many smart, talented SEOs and Flash designers will figure out the answers. It’s certainly a game changer for the industry because it expands the playing field significantly. We are now competing against millions of “new” sites that were once thought of as unfortunate victims of emerging SEO best practices.

It’s no secret that most Search Engine Marketers really enjoy the challenges and rewards of Search Engine Marketing. It’s the thrill of the chase and the hunter instinct that motivates us to pursue customers or prospects online, learning from past efforts, and refining our strategies to increase our effectiveness over time.

We are internet fishermen.

The author fishing on Captiva Island, FL.

We sit just off the banks of the sea of information that is the internet. We cast a line out with a lure meant to attract our desired target. We hope to reel in “the big one” that we just KNOW is lurking out there somewhere.

Just like successful fishermen, we learn to think like our prey. We study them, figure out where they congregate, learn their habits, likes and dislikes. We understand what motivates them to take action. We know they are fickle creatures that will scatter at the first whiff of an artificial hook.

Just like successful fishermen, we obsessively check and re-check our tackle. Are we sending the right message at the right time in the right place? Is our target hungry for what we are serving or are they likely to pass in the hopes that something tastier comes along? Is our offer presented in an irresistible way? Are we simply splashing around or do we have a legitimate chance of hooking a lunker?

Just like successful fishermen, we have to BELIEVE that the next cast is going to land the fish of a lifetime, the stuff memories (and a few “fish tales”) are made of. If the lure comes back empty, we take note of the conditions that created an unsuccessful attempt and use that information to increase our chances of success on the next cast.

This metaphor could go on forever, but I think you get the point. Whether you are doing Search Engine Optimization (SEO), Pay Per Click (PPC), or both, Search Engine Marketing is about trial and error. It’s about being able to scan the seemingly endless ocean of opportunity and knowing where, when, and how you are going to present your message. It’s about learning how to adapt successful strategies and tactics from one industry or target market to another because just like lakes, no two are alike. Fish respond differently based on their geography, history, culture, species and other environmental factors.

Anybody can throw a line in the water and get lucky by waiting long enough. Good internet marketers know that enduring success is not random, but rather the byproduct of preparation and opportunity.

But hey, a bad day of fishing is still better than a good day at work. Are you sure you are fishing where the fish are, or are you merely bobbing around?