As I mentioned in a previous post about the numerous reasons for organic search engine ranking discrepancies, there are a wide variety of tools that can help simplify the process of checking a site’s rankings for multiple keywords across the primary search engines. In this post I will explore four methods for tracking rankings and compare the features, costs, and results of each.

Methodology

I tested the same set of 78 keywords on Google, Yahoo!, and MSN using each of the four methods listed below. The destination domain was AmericanAqua.com, a Michigan-based small business that provides a variety of services including bottled water delivery, salt delivery, water treatment, and water analysis. The keywords selected were a mix of branded terms, product/service terms, and geographic terms. As a control, several broad, generic keywords were chosen where the company has no supporting content on their site and thus would not expect to be ranking. Only the first listing for the destination domain was counted, so if the site was listed in positions 4 and 12 for the same keyword on the same engine, only the position #4 ranking was counted in the results.

In order to control for as many variables as possible, the data was collected in a 4-hour window from the same computer, using the same browser. I was not logged in or cookied by any of the engines at the beginning of the data collection period. The automated rank checkers were set to query no faster than once every 5 seconds to avoid triggering the engines’ anti-scraping mechanisms. Only the first 30 organic results (3 pages) were scanned because let’s face it, if your site doesn’t appear in the first 3 pages you might as well be invisible.

The Contenders

• Web Position 4 is a feature-rich software package that was launched by WebTrends in 2005. Judging by the age of the content and “latest” press releases on their site, there hasn’t been much activity or improvement of the software since then. The software is available in “Standard” and “Professional” packages. Only results from the Professional version are included below.
• SEOBook’s Rank Checker Firefox Extension is a free product offered by Aaron Wall. Once downloaded, the tool itself lives within your Firefox browser as an extension and does not require any additional software.
• SEOMoz’s Rank Checker is another web-based tool offered by the SEOMoz.org team. Unregistered users can run a limited number of queries per day for free, but premium members get unlimited usage for a monthly fee.
• Manual Rank Checking is done the old fashioned way…with a keyboard, mouse, and a spreadsheet. No bells and whistles, just you and a mild case of Carpal Tunnel Syndrome.

¹Web Position 4 Standard edition is $189 and performs all of the same organic rank checking functions. The Professional edition has a few useless add-ons that are not useful for experienced SEO’s or no longer considered useful, i.e. search engine submissions.
²SEOMoz allows registered, non-Premium members to check up to 5 keywords a day for free. Unlimited usage of their Rank Checker tool requires a $49 per month Premium membership, although discounts are given for 6 and 12-month subscriptions.
³For some reason there is an arbitrary 45 minute delay in running Web Position reports on Yahoo after about 50 keywords. Total running time was 84 minutes, but 45 of those were just waiting around.
⁴This probably would have taken much longer, but I didn’t bother checking the ranks of about 150 keywords (50 KWs * 3 engines) that did not show up in any of the other tools.

The Results

Despite the various data collection methods and reporting tools, all 4 data collection methods returned the exact same rankings nearly 72% of the time. Where they were off, the average standard deviation was only 0.84, meaning the difference between collection methods was off by less than 1 position on average. That indicates that the tools are consistently pulling very similar rankings.

However, it should be noted that of the 78 potential rankings on each engine, at most we saw 29 keywords ranking at one time. When the non-ranking keywords are removed from the previous calculations, we only see the same results from each tool 35% of the time with a much higher standard deviation of 1.95.

The complete set of data and calculations are stored in this Google Spreadsheet for you to examine.

Interesting Findings

The following tidbits stuck out while collecting and analyzing the data:

• The four rank checkers agreed most often on MSN rankings. Of the keywords that ranked, the standard deviation on MSN was only 1.08, compared to 1.99 on Google and 2.77 on Yahoo.
• Yahoo’s results were the most inconsistently reported by the four rank checkers.
• SEOBook’s Firefox Rank Checker was the fastest, but missed a few rankings than the other tools found.
• SEOMoz’s tool only allows one query at a time, which slows down the data collection considerably for larger keyword sets.
• SEOBook’s Firefox Rank Checker managed to delete one word from the file export, which caused a few minutes of working backwards to determine what was dropped. I ran that one again and it was fine.
• SEOMoz’s tool reports results as “Found on page X, position #Y”, which slows down the process even more to manually convert to an exact position out of 30.
• Many of Web Position’s additional features are no longer selling points and are practically worthless, such as their search engine submission tool. The only compelling features are the ability to run multiple keywords across many engines and the data trending and export functions.

Conclusions

Given the similarities in results between all four methods, I recommend SEOBook’s Rank Checker Firefox Extension because it is free, fast, can handle multiple keywords across the top engines, and has sufficient data export capabilities. It won’t trend your results over time, but a simple Excel spreadsheet or database can make some great charts and graphs from the raw data. Each rank checking method has their own pros and cons, but Aaron’s tool has the right combination of cost (free) and features (just enough to get the job done, without too many bells and whistles). Other tools have more robust feature sets but are too expensive, or are free but too time-consuming and limited in functionality.

None of the organic search ranking tracking methods proved to be dramatically more or less accurate than the others. There are many external variables that could influence rankings by a few positions or more, and these are likely the main cause for the discrepancies between the various tools.

These data only represent rankings at one point in time. The best way to determine the effectiveness of your SEO campaigns is to measure rankings across multiple engines at regular intervals so you can examine the trends. Do not put too much emphasis on any one data point, since they are very likely to fluctuate over the short term. Instead, pay attention to the longer-term trends to look for signs of improvement in your rankings.

As with any software decision, you must draw your own conclusions based on what is important to you. What features do you think are lacking from today’s rank checkers? How should they be extended to be more useful? Which do you use, and which are your favorites?

I wasn’t going to blog this at first, but after reading Wired’s article on 37signals I felt I should post my experience with Basecamp. I like their attitude towards small businesses and focusing on usability rather than feature bloat, so they get a free plug.

Like most business owners, I am always looking for more effective ways to manage my daily list of to-do items and longer term milestones for a variety of projects. Currently, I rely on a single legal pad of paper and a whiteboard to manage my daily, weekly and monthly lists. This works very well for one or two clients or projects but it doesn’t scale well. Additionally, it makes it hard to keep a running list that spans weeks or months with different projects coming in and out of focus at any one time. There are no backups and no security measures in place, but I challenge anybody to make sense of my chicken-scratch handwriting. As limited as this system is, it has its advantages. It is portable, always on, doesn’t require an internet connection, fully customizable on the fly, and is essentially free.

For the last couple of weeks I have been experimenting with Basecamp, a low-cost project management tool available from 37signals, the creators of the Ruby on Rails web app programming platform. At first I loved the easy way to jot notes, to-do items and milestones and separate them by project or client. I soon realized that as a one-person company I didn’t need the sharing and collaboration tools (yet). So, it basically became an online extension of my legal pad and white board. I did appreciate the integration with my existing Google Calendar so I could view my items alongside my daily agenda. I signed up for a mid-tier package at$24/month, although cheaper and free options exist.

However, the downside of online project management software is that it is not conducive to spur-of-the-moment note taking or stream of consciousness brainstorming, especially when I am away from a computer, on the phone or in a meeting. I found myself going back to the legal pad for these activities, then transcribing those notes to Basecamp. That turned out to be much less efficient than just tracking the activities on paper to begin with. There’s something satisfying and reassuring about putting something on paper rather than storing it in “the cloud”.

At the end of the 30-day free trial period I determined that Basecamp was not generating enough efficiencies to justify the $24/month fee. The free version is attractive but doesn’t alleviate any of the overhead associated with using online tools versus old-fashioned offline tools.

However, I will leave the door open to re-evaluating the tools when I need the collaboration tools for sharing files and delegating tasks to employees or partners. Also, I will probably experiment with some of 37signals’ other offerings, like their CRM, intranet or group chat tools.

Dear Comcast,

By the time you read this, I will have already canceled my Triple Play account. Don’t bother trying to find me…you already know where I live. I couldn’t put up with your frequent internet outages anymore. I called you several times and somebody different picked up each time, making promises I knew you couldn’t live up to. You see, my cable internet connection is my lifeline to work, friends and family. As a search engine marketing consultant, my job is next to impossible to do without access to the innerwebs. I think we should see other people.

What’s that? The rolling service outages aren’t your fault? Just another lame excuse to cover for the fact that you have been jacked into by all of my neighbors too. Don’t think I didn’t notice the 3-4 times a day that we failed to connect, always at the most inopportune times.

We did have some good times, didn’t we? Your cable TV and land line phone services were fine. Oh yeah, except when the power goes out and you disappear again. And you were a pretty cheap date. The $99 monthly promotional charge was almost too good to pass up…until I realized that you were about to raise it to about $130 in another few months. For what? Because we would have been together for a year at that point? Are you trying to drive a wedge between us because you are afraid of commitment, or do you think that you have me wrapped so tightly around your little finger that I couldn’t dare leave you? Is that how you treat a faithful partner?

I need somebody more reliable. Somebody that I can count on to be around when I have deadlines to meet and internet-enabled conference calls to sit in on. Somebody that won’t get more expensive over time. So, Comcast, I’m moving on to AT&T’s new U-verse system starting next Wednesday between 12 and 2. They can offer me what you can’t - a faster internet connection that won’t flake out at the first sign of stress. Plus, they offer just as many channels so I know I won’t get bored and I don’t even have to pay extra for a DVR or multiple receivers. To top it off, I hear they are very low maintenance so I anticipate a long, healthy, mutually satisfying relationship.

Surprisingly, you took the breakup pretty well. I expected much more of a fight and promises to be more faithful and cheaper. It was too late for us at that point. You got your revenge by pulling one last trick…disabling my internet service the moment I hung up the phone. I thought it would take you at least a few days to pack your things and go. Now I’m left with nothing until AT&T comes around next week. I wish I had planned that a little better.

Sincerely,

Andrew

P.S. Maybe I’ll see you around sometime. Can we stay friends? I think you have some growing up to do, but some day you will make somebody very happy and I will be just a distant memory. Who knows? We might have a future together at some point, but for now I just need my space.

Has anybody been using Yahoo! Mash recently? I mean really using it. I know several people that have created accounts just to check it out, but less than a week after the initial launch I noticed that all activity amongst my “friends” has ground to a halt. It’s been a few weeks since then and still nothing. The official Mash blog is not helpful and doesn’t appear to be generating a whole lot of buzz.

I wrote a brief post about my first few days here, and to my knowledge none of my initial concerns have been addressed. The Gmail address book import feature STILL doesn’t work, and to date none of my Yahoo! Mail or Hotmail contacts have registered for Mash either. In fact, I did an informal survey of friends and family that aren’t connected to the internet marketing or search engine marketing community and none had heard of it. The most common response was, “Even if I knew about it I wouldn’t sign up because all my friends are on Facebook.” Okay, a few said MySpace also. Even if they did sign up for Mash, I couldn’t find them anyway because there is no friend finder or people search yet.

This can’t be the type of reception that Yahoo! was hoping for. Is it a soft launch before they drop a huge bucket o’ cash on an advertising or viral marketing campaign? Are they frantically upgrading the features and functionality like they claim? The more I think about, the more it seems like it was launched while only half-baked. The official blog posts are few and far between, and don’t really help generate a lot of excitement.

I’m not going to declare it dead yet, but each day that goes by represents a missed opportunity to build excitement and generate buzz for a platform that has a difficult road ahead.

I was recently invited to join Yahoo’s new social media platform Mash. I was a little hesitant at first because I am busy with my recently-acquired Facebook profile and just canceled my MySpace account because of too much spam and irrelevant clutter. But, because I wanted to see what Yahoo! would do to differentiate themselves I jumped right in. At first glance your profile page will appear very boring and unsatisfying, especially if you are used to other, more advanced social networks. With nothing else to do, and no friends yet, I started adding as much information as possible and playing with the settings. More on that in a minute.

One of the first things I noticed the default setting that allows any of my friends to alter the appearance and content of MY profile page. No thanks. I appreciate what Yahoo! is trying to do by incorporating deeper connections between friends, but my profile is set up the way I like it, thanks. I tend to trust my friends, but the thought of the kind of annoying damage they could cause around 2:00 AM or while I’m on vacation was enough to make me disable that setting immediately. My cynical side assumes it is meant to keep people coming back more often to make sure nobody has mucked up their hard work. It’ll be interesting to see how people embrace that feature.

That being said, Mash does give its users much more flexibility than Facebook (although not as much as MySpace) over their profiles’ appearance. If you are CSS-savvy you can have almost unlimited flexibility, although changing the standard background image, module color and text color are probably enough for most users. I actually like this feature, because my profile can be changed instantly to reflect my mood, team spirit or any other color scheme I choose. Again, I’m not sure I am ready to give up this level of control to my friends.

Hmmm…friends. This is where I ran into problems. The kind of problems that might deter an unmotivated early adopter from ever returning. Like most social networks, they can mine your existing Yahoo! Mail, Hotmail or Gmail contact lists for people that have alredy signed up for Mash. Or, you can choose to send invitations to anybody that is not already part of the network. As of this morning, none of my Yahoo! Mail or Hotmail contacts were using Mash (most use Facebook and/or MySpace) and the Gmail address book import wasn’t working. Just as well, I wouldn’t spam my entire list of family and friends anyway. I’ll try again in a few days to see if anybody I know has tried out the new social network, but I don’t think it has gained enough popularity or buzz to break into the mainstream yet. There are plenty of ways to invite friends via email, but no way to search all Mash users for people in your area or that share similar interests. This feature could use some work.

Another similarity between Mash and the other networks are the use of modules. Nothing fancy here, they can be dragged around the screen just like Facebook. Thankfully the Module Gallery isn’t completely barren. They did have the foresight to add some basics like Flickr photo sharing, RSS feed displays and silly games. They promise to open it up to module developers in the next few weeks, but I’m sure we won’t see them built as fast as they are for Facebook. All signs point to no revenue sharing, so most widget developers might be dissuaded by the lack of profitability.

All in all, after about 90 minutes of experimentation, I don’t see any reasons to abandon your Facebook or MySpace networks. Other than the ability to alter the appearance of your friends’ profiles, there is nothing revolutionary about Mash. Hopefully they have more planned for the future. I’m sure with Yahoo’s resources it will evolve into a decent-sized network, something on par with Google’s Orkut. Perhaps they can dominate niche markets around the world, but I don’t see enough differentiation or buzzworthiness to de-throne the category leaders here in the U.S.. Not yet anyway. It’s too early to tell, but I wouldn’t recommend sinking any resources into Mash at this time.

I would love to hear your thoughts. What do you feel is different, unique or better about Mash? What can be improved? Where do you think it will go? Leave a comment or contact me if you need an invite.

[UPDATE] Yahoo! has launched a blog to support Mash and allow for communication from their Mash lead, Will Aldrich. The obligatory “welcome to Mash” post can be found here.