Illustrated History of Google’s Evolving Algorithm

We know that search engine rankings change (a lot). We know that each update moves some sites up in the rankings while others fall from grace.

But even though this is old news, Aaron Wall at SEOBook.com frequently rants against algorithm changes that affect large numbers of sites (especially his own) while leaving some other offenders unscathed.

Regardless of your opinions, his point is valid. The algorithmic updates sometimes seem to arbitrarily affect some sites more than others with similar characteristics.

But, as Google has always said, it’s their algorithm and they can do whatever they want with it.

Rankings Don’t Matter

The smart marketer doesn’t rely on rankings alone to determine if their SEO efforts are successful. In fact, I recommend NOT relying on search rankings. Check out this post on the SEO metrics that I feel are more relevant than rankings.

Collateral Damage from Algorithm Changes

Aaron’s infographic gives an idea of the “collateral damage” caused by Google’s algorithm changes. Again, you may not agree with everything in it (I sure don’t), but it’s a great visual representation of how the algorithm has evolved.

(click for full-size version on SEOBook.com)

Google's Collateral Damage.

Infographic by SEO Book

Hidden Text Rumors Refuse to Die

Can you see me now?

It’s easy to get caught up in the “state of the art” SEO world when you do it for a living. I got this email from a friend and was instantly catapulted back in time about 5 years:

I’ve been told that KEY words that search engines use to find our site can be hidden.  Meaning that these words are not visibly seen on our website, but the’re still recognized by the search engines.  The key words that I would want listed are the names and addresses of all our competing dentists.

After I regained my breath, I quickly fired back an email that said, “And I heard that cavities are caused by worms,” and included a link to the history of dentistry Wikipedia article.

I couldn’t resist a little professional humor, but also included the following, which hopefully will help dispel the rumors about hidden text on a website and reinforce that it is a BAD IDEA. Hidden text used to improve a web page’s ranking by tricking search crawlers into thinking the page contained more content than users could actually see.

There’s a lot of misinformation floating around the internet about Search Engine Optimization. Hiding text was effective until about 5 years ago, then the search engines “got wise” about how people were using it to manipulate search results. Google’s guidelines are pretty clear, and the penalties are too severe to risk testing it:

From Google’s official guidelines (emphasis mine):

If your site is perceived to contain hidden text and links that are deceptive in intent, your site may be removed from the Google index, and will not appear in search results pages.

More here: http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=66353

Five Web Tools That Triple My Productivity

Feast or famine.

When it rains, it pours.

Make hay while the sun shines.

No matter which idiom you want to apply, I’ve been thankful for the recent uptick in new clients and expanded roles with existing clients.

For no other reason other than that I haven’t blogged in a while, here’s are the tools I’ve been using to dow the work of 3 people without having to add to the payroll:

AdWords Editor – PPC Management

adwords editorGoogle’s wonderfully powerful tool for creating and managing AdWords accounts in a desktop application. The AdWords web interface is great for generating reports and minor changes, but nothing beats AdWords Editor‘s bulk edit and copy/paste capabilities.

I launched a new AdWords campaign for a client last week with over 600 keywords in 100+ ad groups. After a solid week of keyword research and writing ad copy, it only took me 3 hours to actually create and upload the campaigns using AdWords Editor. I don’t even want to think how long this would have taken using the online interface.

Freshbooks – Online Invoicing

freshbooksInvoicing clients is a necessary evil. Yeah, it’s fun to watch the money roll in but Quickbooks was nightmarishly clunky, slow and manual. It took about 10 minutes to fire it up, log in, create an invoice, save it as a PDF and email it to a client.

About 5 months ago I switched to Freshbooks, an online system for generating and tracking invoices. Now, creating a new invoice takes less than 2 minutes and my clients get them instantly via email. Instead of logging in to see who might be delinquent (or close), I can track who has viewed their invoices and automatically send a reminder before it becomes overdue.

It even handles time tracking and expense tracking by client to make task-level invoicing really, really easy. And kind of fun.

Unbounce – Landing Page Testing

unbounceThere’s nothing worse than knowing your landing pages are not converting as many visitors as possible. You’re leaving money on the table, plain and simple.

Unbounce is an idiot-proof system for designing and A/B testing multiple landing pages at once. As more conversions take place, the winning combinations of images, text and calls to action become clear.

Use the learnings to create smarter, more efficient landing pages that convert increasingly more visitors into buyers, email subscribers or downloaders. Best of all, you can directly add new leads to many major CRM and email marketing programs (like MailChimp).

MailChimp – Email Marketing

mailchimpStaying in touch with customers is a must. They are more likely to respond to offers from companies they’ve dealt with before, so why not reach out to your previous or current customer base with a new offer or information that adds value to their day?

There are a lot of ways to communicate with people online these days, but nothing is more efficient or effective than email marketing. MailChimp is the best solution I’ve found for building and managing lists, designing email templates and analyzing the results.

Take the time to reward your best customers. They’ll reward you back many times over.

ActiveCollab – Project Management

activecollabI’m not a big fan of recurring monthly subscription fees for web-based software unless there are simply  no other suitable options. Many people love BaseCamp for online project management but I actually chose ActiveCollab because it offers 80% of the features at a one-time fixed cost.

It’s not perfect and not nearly as sexy as BaseCamp, but it works just fine for small teams and agency/client collaboration. It’s certainly worked to keep me on task and on schedule. And after only a year, it has basically paid for itself in saved monthly subscription fees.

Help Me

What tools/services/software do you use to improve efficiency and generate better results for your company or clients? I’m always looking for ways to save time or money. With a new baby and lots of new client work, it’s getting harder and harder to find the right balance between the two.

My Take on Google Instant

I have been flooded with questions from clients, colleagues and friends since Google’s announcement of Instant search. If you’re not familiar with Instant search, check out this quick video I shot before the official announcement. Or, of course there’s always a highly polished video from Google:

The new reality of SEO and PPC?

Personally, I found Google Instant distracting at first. After a couple of days, I am pleasantly surprised that it seems useful in a lot of cases where I’m not quite sure what I’m looking for. The general use case is somebody starting with a broad search (i.e. “flowers”) that will benefit from seeing suggested search queries based on their predicted intent (such as “flower delivery in richmond, va”).

That being said, the implications for SEO and PPC marketers are currently being discussed and analyzed throughout the industry. Here are the key themes that I believe are becoming the new reality:

1. More concentration on “head” terms. These high search volume, highly competitive, expensive keywords are likely to show up more often in Google’s suggested or predicted queries. Our keyword research needs to take these suggested or predicted queries into account so that we can optimize websites and PPC campaigns accordingly.

2. Long Tail keywords will be less productive. As Jeremy indicated, I believe most searchers will be distracted or tempted to click earlier as the search results change in real-time. This will likely lead to more clicks on head terms and more search query refinements.

3. Ultimately, relevance is still key. Organic and paid search rankings have not changed with Instant. No matter how the User Experience of search evolves, a well-thought out SEO strategy will include tactics to improve a site’s relevance to the keywords people are most likely to search for. Even if you rank #1 for a popular query, a poorly written Title tag or call to action are going to diminish the likelihood of people clicking on your organic search result or converting on your site.

4. Google will get richer. The logic behind this is obvious, since the head keywords are typically more expensive on a Cost Per Click basis. When it comes to their golden goose, Google really only makes UX changes like this that will improve their bottom line.

5. Brands matter more than ever. Large brands stand to gain considerably because Google inherently trusts them more. Over the last year, we’ve seen Google start to rank well known brands higher than lesser known but potentially better-optimized sites. This evolution is not debated in the industry, but the pundits and experts disagree on whether or not this actually improves search results and the search experience for the average Google user. I tend to believe that less variety and choice leads to lower quality search results.

6. Instant doesn’t mean permanent. Google is well known for launching a major UX change or search feature only to roll it back if user acceptance is not high and/or it doesn’t have the intended outcome (higher profitability for GOOG, better user experience, ease of use, etc.) Remember how short-lived Google Wave and real-time search results were?

7. The proof is in the analytics. We won’t know the full impact of Instant on SEO and PPC until marketers and analysts have had time to gather and interpret any changes in site analytics or campaign metrics. I expect to see preliminary results being reported as early as next week but a full understanding may take weeks or months, especially when seasonality is considered.

8. Remain calm. We’re on the leading edge of this change and word travels fast in the SEM industry. We are better off taking a wait-and-see approach to making changes rather than jumping into the unknown. Even waiting a few weeks or months to modify an SEO or PPC strategy will still likely give you an advantage over the vast majority of competitors that will be slower to act (if they act at all).

I hope this puts your mind at ease. It’s an exciting change to keep an eye on, but the end result will ultimately depend on whether or not the average Google user accepts the changes or not.

Google Streaming Search Results

Just for fun, I thought I’d post a screen capture of Google’s new “streaming” search results that update in real time as you modify your search query. It’s best viewed in full screen or HD mode.

As with the search suggestions that all the major engines use now, this will likely change SEO for most marketers as we are going to have to contend with more “eye candy” that could distract searchers from our desired search results.

While real-time search suggestions may add some value to people searching for broad terms, real-time search results are distracting and confusing at first. I’m not sure if I’ll get used to them or not, but it does seem to add some amount of unnecessary distraction.

What do you think? Have you seen these results in the wild yet?

PPC in 5 Years: An Intractable Problem

Every now and then I like to get out my crystal ball and think about where I see the search industry heading. It’s impossible to predict because every day brings new changes, but there are certain trends that are likely not going away.

The way I see it, there’s an intractable problem between the breakneck pace of innovation in the search marketing industry and advertisers’ abilities to take advantage of the new hotness. I’d love to know if you agree, disagree, or have any thoughts to add in the comments!

PPC Advertiser Challenges

Ad targeting options are becoming so varied and complex that less sophisticated advertisers are missing a lot of opportunities to improve efficiency and are unable to effectively compete. Given the inherent complexities, the vast majority of advertisers will not be able to remain efficient even with more advanced and automated bidding optimization tools at their disposal.

Less sophisticated advertisers will be slowly wither away at the hands of geekier competitors.

The primary disconnect is lack of conversion data. In a mathematical model where profit margin and Cost Per Conversion is critical, less sophisticated advertisers will either slowly bleed to death or decide to advertise elsewhere.

The Intractable Problem

Here’s the rub – the PPC space is going to be so competitive and saturated that only the smartest mathematicians and analytics gurus will be able to turn a profit. Why? They are the only ones that can effectively manage a PPC campaign to a target Cost Per Conversion that allows for a predictable, manageable profit margin. Everybody else is paying too much for customers and that is not sustainable in the long run.

If you think click costs are expensive now, just wait until adoption rates are higher and more advertisers are using more advanced tools to squeeze your margins.

Sure, more advanced and automated bidding tools are becoming available every day, but the level of understanding and experience to configure and manage them is still high. No matter how smart the black box is, it will still always require a human to make decisions based on the data provided and factor in offline inputs.

Who Should Be Worried?

PPC advertisers have the most to lose by spending money with little to no hope of generating a positive return. That’s a lesson quickly learned (hopefully) and marketing dollars can be reallocated to a more profitable channel.

Secondly, I believe Google, Yahoo/Bing and other PPC advertising engines have to do a better job of keeping a level playing field. Their real-time auction pricing models and profitability depend on having a steady stable of advertisers bidding on as many keywords as possible to drive up prices.

Just as water always finds its level, fewer, more sophisticated advertisers will eventually find the “ceiling” price for a click and would be forced to find operational efficiencies and higher conversion rates to improve their margins.

Am I Right?

Who knows? The PPC industry will be different in 5 years, that’s a given. But if these trends continue to hold, I think that only the smart will survive.

SEO Attitude Adjustment?

My friends at RedShift and I started an email dialogue about a MediaPost article that proposes that SEO consultants need an attitude adjustment. I gathered my thoughts in a blog post to avoid a long email thread, so here goes.

From the article:

Mike Moran, chief strategist at Converseon, rallied the troops at the Search Insider Summit in Captiva, Fla. Friday to not only become transparent, but focus on serving consumers rather than getting backlinks or appearing on the top spot on the first page in Google search results. Don’t forget the PageRank, but think more about answering consumer questions.

The Glass is Half Full

Half full or half empty?

I had seen the article and completely agree. Search Engine Optimization is increasingly focused on content marketing and promotion (as it should be) to serve consumer needs and help them accomplish their desired tasks online.

But let’s not ignore the reason SEO exists in the first place. There still needs to be plenty of attention paid to the basic blocking and tackling of SEO. Social media, PR and content marketing folks parading as SEO’s frequently miss the point and blur the lines between two distinct yet complementary skillsets.

Search Engine Optimization (in my mind) is a more technical specialty that includes some very geeky “behind the scenes” tactics that are often overlooked or ignored by people focused solely on creating content, including:

  • Proper server configurations
  • Transitions to new domains
  • Site redesigns
  • URL and duplicate content issues
  • Search crawler accessibility
  • File structures and site architecture
  • Information architecture
  • Link building

Again, content creation and on-site SEO go hand-in-hand and should never be separated. I’m just wary of people pretending the technical components of SEO are no longer necessary or important just because they haven’t taken the time to learn them.

Database of Intentions Revisited

John Battelle is a smart man. Instead of focusing on the micro-level changes in the search industry, he thinks big. His book The Search is one of the foremost books about Google and the rise of search engines as an integral part of everyday life.

His original premise is that Google represents The Database of Intentions:

The aggregate results of every search ever entered, every result list ever tendered, and every path taken as a result.

It’s a big idea with huge ramifications, but pales in comparison to the revised 2010 Database of Intentions:

  • The Query
  • The Social Graph
  • The Status Update
  • The Check-In
Database of Intentions

Revised Database of Intentions

Read the original post and spend a few minutes thinking about the implications for your business. It’s sure to spark some interesting conversations and thought in the coming weeks, months and years.

I am looking forward to hearing John expand upon these ideas. Big thoughts like this are why I love my job.

Finally, Some Useful SERP Click Statistics

There’s a lot of FUD (Fear, Uncertainty and Doubt) surrounding click statistics and metrics for search results pages. Reliable data is hard to come by when trying to answer questions like these:

  • What percent of searchers click on a result on page 1 of a search result? Page 2?
  • Is there any value to having organic search rankings on page 2 or page 3?

Enter iCrossing. A recent study (PDF) concluded that 95% of website traffic from organic, non-branded queries on Google, Yahoo and Bing originated on page 1 of the results.

Click Distribution on Search Results Pages

Click Distribution on Search Results Pages

This is big. It means that unless you rank on the first page for non-branded (not your company name) queries, you are fighting for scraps while your competition enjoys access to 95% of your prospective customers.

Trends and Previous Findings

Naturally, the search engines don’t (normally) release this data and there are few search marketing agencies large enough to perform a statistically significant study on their clients’ data. However, there are two previous reports that we can refer to for benchmarking:

  1. AOL’s 2006 data release, while widely criticized for violating personal privacy, indicated that 90% of organic, non-branded clicks were on page 1.
  2. iProspect’s 2008 study found that 68% of searches resulted in a click on page 1 of the results (up from 62% in 2006, 60% in 2004, and 48% in 2002). However, no differentiation is made for branded vs. non-branded terms in this study.

Even with different study methodologies and data sets it becomes clear that there are two forces at work:

  1. Search engines are getting better at providing relevant results, minimizing the need to click past the first page of results.
  2. Searchers are getting better at defining what they want and using more targeted queries to find it. This is reinforced by other studies that show the average length of a search query continues to grow over time.

So What?

I don’t intend to sell you on the benefits of SEO. Instead, we’ll talk about supply and demand. It is becoming increasingly clear that demand is far exceeding the supply of available links on a search results page. Therefore, the value of high organic search rankings for non-branded keywords will continue to skyrocket as more companies line up to compete for limited space.

Now What?

Take some time to identify the gaps in your keyword strategy. Use some free or cheap rank checking software to get a glimpse of your current rankings (but don’t get hung up on them because rankings change constantly). Now, check your web analytics package to identify the best-converting keywords that you know you are missing out on.

Now, if you notice your site ranking on page 2 or page 3 for some of your high value keywords, you’re most of the way there. Give those keywords some extra attention on your site, build a few links, and watch your rankings jump up to the first page.

Rinse and repeat for different sets of keywords.

Imagination Is Endless, So Are Keywords

Google Queries in 2009

Google Queries in 2009

According to Google’s 2009 search stats, over 1/3 of unduplicated search queries entered into Google in 2009 HAD NEVER BEEN SEEN BEFORE.

Let that sink in for a moment. That’s up from about 25% a couple years ago.

Does your content strategy allow you to take advantage of people’s creativity and difficulty finding the answers to their queries? Or are you stuck with stagnant content that does not evolve and adapt to meet consumers’ changing needs and vocabulary?

But Wait, There’s More…

If you are not concerned about your position in today’s search results, you should at least be preparing for tomorrow’s. The Google ranking algorithm literally changes daily:

  • Number of search quality improvements made by Google in 2009: 540, ~1.5 each day

And if you’re not effectively competing in local search results, consider this whopper:

  • Proportion of Google result pages that show a map in search results: 1 in 13

Do Not Be Alarmed

I share these stats not to scare you, but to open your eyes to the nearly infinite opportunities to attract quality search engine traffic from a wider set of keywords and search channels.

Sure, your boss questions every day why you don’t rank on the first page for “widgets”, but the smart marketer knows that there are still plenty of battles to be won. It’s just a matter of picking the right ones.