seth 09 May 05

You need a robots.txt file.

ward andrews 09 May 05

i have found that other references to a product, especially from popular blogs, appear first when you launch a product, then over time your official link takes over…after a week or two.

Keith 09 May 05

Seems to be in there now. My guess is it’s one of those random Google glitches, a “Google Gremlin” if you will.

;)

I’ve noticed some of my sites disappearing occassionally when I search for the direct URL. In every case, it’s showed up again a bit later.

Not a real answer, but it’s the best I’ve got.

Dave Marks 09 May 05

Oh Boy… At the moment I hate google…

Its quite possibly you are suffering the effects of the Google Sandbox effect like Adam said. (If such a thing really exists) Check out www.webmasterworld.com for lots of info on this. When did you first register the site and put up a holding page? If it was last year, you should be alright.

I too have quibbles about the adwords thing… we have spoken to google on several occasions but they always deny it…

MSN seems to to provide way better results at the moment, I’m really starting to go off google… I have no time for it these days.

seth 09 May 05

alright, so I take it back. I guess you don’t need a robots.txt file.

a while back my site wasn’t getting picked up and a google staffer told me to make sure I had a robots.txt file.

so I put one up, and soon saw my site get crawled. Perhaps it was just a coincidence.

Chris from Scottsdale 09 May 05

I’ve been using Yahoo recently and it’s actually better for localized stuff if you want my opinion. When I want to find a local company it wins hands down.

Peter Cooper 09 May 05

Google, generally, won’t include new domains very quickly at all. That’s why I launch all of my products (RSS Digest, Snippets, etc) all under subdirectories of my main domain name. That’s also why they have high positions in Google for all of their relevant words and phrases.

The day of individual domains for individual products from the same company are very much over if you want to be high in Google nowadays. It doesn’t seem to bother users either.

There is, however, a way to get around it. Put a very basic site (at least a few pages though) up on your new domain a few months before you launch it, and then point a few PR6 or higher links to it. You should be in after 3 months, and you’re ready to roll. Your positions will be mostly abysmal, however. I have pulled this off with a couple of domains, but they still rank poorly.. whereas brand new pages in subdirectories of my core business domain name can totally ace everyone else within weeks! (I’m getting 500 visitors a day from Google already on one launched only a month ago)

I second/third the recommendations to read WebMasterWorld, and, heck, go to their conferences. You meet amazing people and learn amazing things.

And, no, a robots.txt is not required.. HOWEVER, your server must either respond with a valid robots.txt OR a 404. If you respond with nothing useful (as Instiki used to, and also the RoR main site) then you will not get indexed properly.

Peter Cooper 09 May 05

P.S. I believe subdomains under an existing well-placed domain name will also work as well as subdirectories.

Steve French 09 May 05

I can’t find a cache or a copy of the old Backpack page. Feel free to strike me down if I’m wrong here, but wasn’t it just a “coming soon” type of page that collected email addresses for the mailing list?

From my limited understanding of the way the big G works, I thought they liked lots of text and heading tags. And if all this is the case, maybe they just haven’t gotten the new page indexed fully… yet.

-Steve

Dan Hartung 09 May 05

Well, it’s there — my Google results for “backpackit” have it showing up as #27, though oddly the first result is the terms of service page.

The explanation, though, may lie in the fact that (as Steve French implies) the home page was previously indexed without a TITLE tag — when I do a More results from backpackit.com the home page is in there but all you see is the URL. Having that page up long enough to get indexed that way was probably your downfall. Even if it just said “BackpackIt: Coming Soon” you’d be much more highly ranked by Google. As it is their longstanding deprecation for pages without a title and/or without the site name in the title takes over.

So, scratch the sandbox effect. Though I’m sure it exists to some extent, I don’t think it’s as important as this deliberately-encoded prejudice.

Marc Hedlund 09 May 05

It took about a week and a half for CodeZoo to show up. When it did show up we were immediately #1 for the search ‘codezoo’. This was despite tens of thousands of hits showing up for the term ‘codezoo’ that weren’t in the index before we launched (that is, hits on that term went from under 300 before we launched to nearly 100,000 after we launched, but before our site showed up in Google).

jpack 09 May 05

Well, I see 305 pages (with site:backpackit.com) but I also see PageRank 0, though in an “unknown” rather than penalized fashion.. But that could explain some bad rankings.

I would make backpackit.com rewrite to www.backpackit.com for consistency.. plus it’ll make it easier to see what’s indexed for your main site vs. all those crazy subdomains.

I’m surprised you aren’t putting robots.txt on at least some parts of the site?

Jaime Smith 09 May 05

One thing I noticed is that you appear to be searching on “backpack personal information manager”. However, the word manager, and even more the phrase “personal information manager” does not appear anywhere in the visible text of your home page.

From my experience, Google is more prone to using the visible text of the page (versus meta) for indexing. So, by not having one of your keywords in the visible content of the page you are limiting the likelihood that your page will be found using that keyword. Given that so many high-traffic (read high PR) sites are linking to Google using the phrase “backpack personal information manager” I wouldn’t expect to see your home page listed near the top at all.

All that said, as much as your site is probably being crawled right now, I would expect that a slight tweak to your title and the addition of a few key search phrases to your content would quickly result in a boost in the rankings. You should definitely add “manager” to your title and body content if you expect it to be a highly searched keyword.

JF 09 May 05

Jaime, it’s not showing up with any combination of keywords.

Jason Leister 09 May 05

I agree with Dan H. - It looks like your index was kicked out of Google for not having any content (and probably no TITLE tag).

If you search for the Backpackit.com URL or do an allinurl:backpackit.com, you will notice there is no cached date for the index page.

Shouldn’t be long before you are back in and your index page is back in the cache.

Geoff Hankerson 09 May 05

To expand on what Jaime Smitth says

Loose the graphical text header that says “Getting Organized Couldn’t be easier…” and make this section real text. Use an tag for your header and use a phrase with the words you suspect will be searched. What you have in the tag is good: “Personal and Small Business Information Manager”. Then use a

tag for the text under the heading. Then you can use CSS to make this look as nice (or almost as nice) as the graphical text.

Then the metadata and the body of the text will match. The image is killing you in Google rankings.

Josh 09 May 05

I think it’s time for another Google algorithm update :D

My site has SO many links back to it, but my PR is still 0, and I don’t understand why.

Carl DeCaire 09 May 05

Maybe this has been mentioned, but wasn’t backpackit.com just recently launched? I’ve had sites take up to 3 months to index… others a matter of days. I’ve used the same basic system of robots.txt and meta tags including well structured sites (xhtml/css) and they all have various results.

In my own stats, I’ve watched googles bot for a few months and I’ve gotten 500+50 something hits some months and this month I’m down to 1+2 (the + is how many times its hit my robots.txt)

So this might indicate how active their bots are from month to month.
I bet if you just keep an eye on it, you will see it.

Dana O 09 May 05

I’ve found that listing your site on DMOZ will help get things rolling — since Google pulls aspects of that database in their search results.

It can tricky though, because they make you commit to one category in the directory. There are subcategories, but I’ve myself thinking that my listing covers more than one specific category. The listings boosts I’ve gotten do help though.

Hope that helps.

Tiara 10 May 05

My friend’s official website (she works in the media) took a while to show up on the results - the stuff showing up on the first page were news articles, photo galleries, and my fansite for her. It took about a couple of weeks or so before it showed up on the first page, and now it’s #1.

Also, people who are searching for “backpack” would most likely be searching for bags or travel advice, not the BackPack PIM. Since the only PIM-related “backpack” is yours, and there’s tons more backpack references to bags and travel, it might take a while before BackPackIt.Com ends up on the top few listings.

EB 10 May 05

How are you guys calcualting PR on google results ? As per *jpack* email.

Thomas Maas 10 May 05

You might also want to turn your current h1 tagged bits (3) into h2 (and h2 to h3) and create a new h1 equal to the title tag. Which could still show the image by using an image replacement technique.

A page should have one h1 tag (almost) equal to the title tag.

mark rush 10 May 05

SANDBOX EFFECT - when a new site gets a sudden influx of in bound links it gets put on hold to stop search engine spamming (a deterent) theres no way to beat it but if you type in “backpack -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo -moo” without speech marks you should see a result that is more akin to what it will be like once the sites been reintroduced after sandboxing - its stupid i know but what the hey! a victim of your own success…

Michael Martine 10 May 05

The sandbox is real and can take a month or two. Your AsWords ads appear because the AdWords bot crawls and indexes your site independently of Googlebot’s normal web crawls or deep crawls (the schedules of which Google is very tight-lipped about, but it’s presumed the deep crawl happens about once a month). I just finished reading “Building Your Business with Google for Dummies” so this info is fresh in my mind. BTW, I used to deplore the “dummies” books because of their title, but this is an excellent book about everything you could want to know about optimizing, AdWords, and AdSense.

Anonymous Coward 10 May 05

Try this…
http://www.google.com/addurl/?continue=/addurl

Chris 10 May 05

Which could still show the image by using an image replacement technique.

I’ve heard image replacement is a no-no negative for Google. Anyone have any input on this?

Egor Kloos 10 May 05

Lots of good advice, so not much to add here. Only that you could use ‘nofollow’ on links that could hurt you being indexed. The href ‘sitemeter’ link at the bottom is a condidate for a ‘nofollow’.

I’m sure it’s not killing your business. You have lots of high quality links to the backpack site which are found. I minor detour with a thumbs up testamonial can’t hurt now can it?

jpack 10 May 05

EB - Just getting PR from the google toolbar, i.e. going to backpackit.com and looking at the green pagerank bar.

Actually I’m using firefox and this plugin, but whatever.

tom sherman 10 May 05

Lotta misinformation getting thrown around here.

The Google Sandbox has nothing to do with linking to yourself. Forget that. The Google Sandbox has nothing to do with robots.txt. Forget that.

As a webmaster, you’re completely helpless if you’re in the sandbox. New domains are penalized, and this generally lasts 3-4 months. I’ve read of this lasting longer, and a friend of mine has PR0 on his site despite it being more than one year old. He has more back links than I do, and I’m PR5.

Beware the SEO FUD and snake oil. C’mon now, folks.

Scott Meinzer 11 May 05

You mentioned a while back about how you guys coded the site so it would work with the google web accelerator. Is this actually providing different code to google to do this?

Because google gets pretty mad about that. They will actually check a site from two different user agents to see if the content is different (more so then a date or something), and if so they penalize greatly.

Just an idea.

Todd N. Lokken 12 Sep 05

I wouldn’t be surprised to see an update soon. Todd N. Lokken

John Feeney 27 Sep 05

I, for one, firmly believe in the notion of a google Sandbox.

I created a new website about 2 months ago (www.emeraldtracks.com) and did everything to get a good rank in Google: URL submission, good word density, reciprocal links, created a google sitemap etc etc.

It showed up on Google for a whole week before disappearing again and hasn’t shown up since. Meanwhile, a search of “professional Irish MIDI” gets #1 rank in Yahoo, while on Google it’s not even listed.

I don’t know what’s going on and I think it must be this Sandbox thing. What else could it be?

Kenneth 06 Oct 05

Google Sandbox………..?
Not such a far off concept, considering they are a profit making company who need to reward advertisers.

John.. firstly great site! Have you considered buying Google AdWords? Are you really saving that much money considering the amount of time you have spent on this problem.

If the sandbag theory is true, it is an utter waste of time trying to beat them at their own game. At the end of the day, they control the search engine and seem to constantly change their criteria.

I know that�s not how the west was won, but are you trying to fill a Black Hole with sandbags?

Rich 21 Oct 05

I think there is something screwy going on with Google and OPD ( DOMZ ).

Uncover China Fan 06 Jan 06

I know this is an old post, but how did it go? Were the tips above succesful?

Travelscoop 21 Jul 06

I guess its resolved.

Travelscoop 21 Jul 06

I guess its resolved.