Alright, lets get the obvious puns out of the way: Googlebot needs some natural spider enhancement, Googlebot needs a bigger band-width for your site, Googlebot needs to last longer crawling your site. Those are pretty bad…. but, if you came here for the pen15 jokes, thanks! Now, move on, perv.

What I am going to actually talk about is site architecture and how you might be wasting Googlebot’s crawling bandwidth for your site.

Site Architecture for SEO

How your site is structured is very important to SEO on many, many levels. For example, a proper site architecture allows SEOs to target keyphrases properly, develop authoratitive pages and appease the mighty search spiders.

An ad-hoc site architecture can be difficult to work worth and even more difficult to achieve the results that many sites are looking for out of their SEO campaigns. So, before you jump into your site – plan ahead.

Lets say your are completely re-doing your site. What you want to do is during the wire framing phase of your site’s build out think SEO or bring an SEO into the discussion so you know you have your major keyphrase targets covered. This will make the optimization go smoothly down the road. Also, an SEO will be able to make suggestions as to what is missing so you accommodate expansions now or make the site flexible enough to accommodate this new content later.

Evaluate Your Platform

Take the time to really test out the platform you are considering on using for your site. Focus in on how the platform builds the URLs as you click around. What you are keeping an eye out for is URL-bloat through improper canonicalization of URLs or a general issue with how the platform actually works. This is a major consideration for larger e-commerce and news sites because you never want to see the “Googlebot found an extremely high number of URLs” error.

Googlebot Found an Extremely High Number of URLs

The above error message means you have a serious issue on your hands. If Googlebot is having an issue with the number of URLs your site is offering up for crawling you can bet that Bingbot and every other bot on the planet is probably having the same problem.

Your Whole Site isn’t being Crawled

The error basically tells you Google has used too much bandwidth crawling your site and they aren’t going to waste any more. This boils down to repetitive URLs displaying the same content (a canonicalization issue). Large sites that have tons of pages but each page has unique content won’t run into this error, but it’s large sites that have a dysfunctional CMS that will always get this message.

See Step 2

You may have missed something while evaluating your platform, so circle back on how your CMS is building your URLs. Googlebot has clearly found itself in an area of the site that is continuously spitting out URL after URL that aren’t necessary.

Google even offers up a bit of advice on this too: http://www.google.com/support/webmasters/bin/answer.py?answer=76401

Googlebot can’t afford Enzyte

Google is crawling the entire Internet. Everything! They just can’t afford to spend forever crawling your defunct site in hopes that eventually they will find the content they were meant to find. Having a proper architecture and platform that respects that architecture will help to keep you out of trouble.

No related posts.

<Announcer’s Voice>
Google: search engine; advertising giant. Searching for a way to tap into the hidden strengths that all their search results have. Then an overdose of webspam fighting radiation alters their organizational chemistry. And now when Google grow angry or outraged, a startling shift in the SERPs occurs. The Google Creature is driven by deceptive SEO and pursued by any whistle blowing investigative reporter. [Google:] “SEOs, don’t make me angry. You wouldn’t like me when I’m angry.” The Google Creature is wanted for penalties that it delivers. The Google SERPs were believed to be dead, and Google let the world think they were, until the Webspam Team can find a way to rid them of the spam that dwells within them.
</Announcer’s Voice>

For those of you who aren’t getting the reference, read The Incredible Hulk opening narration.

The Google Creature Evolves

We were warned that Google would be kicking spam fighting up a notch in 2011. However, no one really guessed that they would be taking it from soft plush toy level to monstrous beast eating big brands for breakfast, lunch and dinner! Not to mention in such a short period of time. Which would you rather have on the case plush toy or monster?

Not a Plushy SEO World Anymore

In recent years, SEOs have basically gotten away with murdah.

As Google was committed to growing it’s product lines and bottom line, they were relatively soft on spam in the SERPs. This was an opportunity for SEOs (large and small, in-house and agency) to take a little bit a leeway with some of the Google Guidelines. It also allowed entire businesses to grow out of exceedingly low quality content farming because they can make a buck off of traffic from Google.

But, clearly, this is not the case anymore. Google has lined up big brand after big brand over the past couple weeks and slapped them with penalties or the threats of penalties. The ones I am talking about are JcPenney, Forbes and Overstock. Also, Google has launched algorithmic updates that take a stand against content farms by elevating ability to better detect source attribution.

Update: Information on the first algorithm change in January and now another in February that combined impacts close to 14% of all the searches conducted on Google and specifically aimed at improving SERPs based on content quality of sites.

Against the plushy spam fighting, SEOs were able to take some more chances with link building and other tactics that are in the black and grey areas because they were working and there were no immediate repercussions.

Under reinvigorated Webspam monster, these brash tactics are going to get you in trouble. The SEOs that think they have found a “loophole” for their client or organization to manipulate the results are going to quickly learn that it is not the direction for long term success (for the client or your career).

Taming the Beast

Stick to SEO techniques that fall within the guidelines. There is no loophole that will ever go completely undetected by Google, your bad SEO foot prints will be found and you will pay big time in the SERPs. By playing by the rules that Google has set up you will completely avoid having to tangle with the Webspam Team’s monster.

These are my hats and wearing them I have only had success.

No related posts.