More recently I’ve started inserting various bits of text into headers of sites using jQuery and AJAX – things like contact details etc. This way when the site’s crawled, non essential things like the phone number or ‘about us’ paragraphs that are sometimes common towards to top of every page of a site aren’t indexed and seen on every page by google, thereby diluting the genuine content of a page or been flagged as duplicate content. On the homepage or about us page I leave it in the original source so it’s indexed appropriately, but for the rest of the site’s pages I insert it dynamically by AJAX when needed. Again, I’m not sure how much of a difference that makes but it just seems like good practive to me?

Although you’re correct that page structure shouldn’t should be dictated by what’s right for users, not what’s right for robots, when the difference between two (or more) options is intangible for the users but makes a difference to the robots then, absolutely, you should optimise for the robots.

What it’s worth bearing in mind is that (a) the robots are designed to emulate the behaviours of humans (although how well they achieve that is variable both over time and between engines, of course), and what works well for humans often works very well for robots too, and that (b) on an awful lot of sites, the primary traffic source is the robots in the first place: if nobody can find your page, it doesn’t really matter how it’s structured.

My personal experience suggests that building human-oriented content-focussed well-structured pages works extremely well as far as search engine spiders are concerned, although there’s always plenty of fine-tuning that can be done for both humans and robots, and that’s all this debate really centers on: if all you’ve got left is content first, navigation second, or vice versa, then you’re going to do pretty well for both audiences whichever you choose—but that doesn’t stop you wanting to improve the experience of either, or both.

In many ways, I think this obsession with what the googlebot thinks of us, is indicative of the untidy and disorganised way in which web content is made available, these days, and no one in particular is to blame, for that: why try to order your content, when your visitors all arrive (and depart) as a result of something they typed into a textbox? The success of the search engines both created this effect, and help to drive it. As a result, it is often easier to find anything relevant than it is to find something specific. In fact, the more specific the problem, the more likely you are to find a solution, if one exists (and, of course, the less likely you are to visit the site you found it on, ever again). You don’t even have to spell it right! I suspect Google’s list of misspellings of ‘Ubuntu’ probably run to several pages: not bad, for an obscure Xhosa word for a concept that cannot be stated in plain English.

But, there you go: sometimes it’s just eaier to put a name to a problem, than it is to put a name to a face.

Interesting, as this is, however, it should not cloud our perception of the fact that we build for other people, and not for robots. A robot will remember your name because it has been programmed to. if you’re important enough, it’ll even correct you, if you spell it wrong. It cannot help you with this, though:
“brummie+feller%2C+talks+about+accessability+a+bit%2C+likes+the+odd+beer”
But who knows? I might actually want to read what Bruce was writing about in September 2003. I can’t get there by tabbing, though ) (hmm, I can see the campaign slogan, now: ‘Give me content, or give me carpal tunnel syndrome!’). In fact, ironically enough, he makes us use his search facility. Maybe we should all just ditch our site navigation just and create a network interlinked of search boxes?

If you are relying on ‘tabbing’ to navigate, (as many of my friends do) it’s nice to have, at least the main navigation, at the top of the page structure. If it’s at the bottom and you have a lot of ‘in-page’ links, you have to step through all these to get down to the main navigation, unless, as was mentioned, you just assume the navigation is always at the bottom and jump straight to the end of the page.

That’s not true.

If you’re working with very competitive terms (where subtle changes can make more difference), having those terms early in the source makes a difference.

It follows that it also makes a difference for less competitive terms, but is less of a factor.

I suspect that’s not true, but have no evidence, just a hunch.