Web Log of Greg Mathews

Before I start please let me clarify, when I ask the question ‘What is the point of a Google Sitemap?’ I am refering to the Google Sitemap and not an HTML sitemap you have on your site for your visitors to browse the site easier.

I have always beleived that a Google sitemap is of use when launching a rapidly growing site, I do not think they are beneficial to sites with last than half a dozen pages or so.

I launched a site back in October 2006 and regularly submited a sitemap containing all of the pages on the site, this sitemap was changing on a daily basis as the site grew. Within 10 days the site was indexed and showing well in the Google SERPs, (Search Engine Results Pages) far quicker than I ever expected or would have seen without the use of a Google Sitemap in my opinion.

Things went well for the first couple of months until Google appeared to Drop the site from its index in January, but this appeared to be a glitch and within a couple of weeks the site started to appear in the SERPs again.

Then this month (March 2007) a similar situation happened, although using the site command on the domain in Google still showed over 100,000 pages in the index only about 50 where included, the rest never appear, not even in the Suplimental results.

When I looked at the SERPs for my site it appeared the only pages indexed where the html sitemaps, which are just pages of links to other pages on the site. When I checked my server logs it these where also the only html pages being crawled by the Googlebot. Other files Googlebot was also crawling where for a cgi script that the html pages had in the form of <script ‘type=’text/javascript’ xsrc=’url to script’> which is just a logging script that returns nothing.

From this I am assuming that Google is now no longer using my Google Sitemap but instead are doing a more thorough crawl of my site by following the links, both internal and external. This is a good thing in that it means real pages with links to them will be indexed and obscure pages will not, however I do have the following questions :

1. Did Google need to remove all of my pages from the SERPs before changing the method which they use to find pages on my site? this has cost me about 75% of my traffic/income.

2. Why oh Why is Googlebot following javascript links? each of my 100,000+ pages has the same javascript link but each with a different parameter being passed to it in the url, this means that the Google bot is crawling the same script 1000’s of times. I have blocked Googlebot from crawling this script now via the robots.txt file but this could have implications as the script is used for other tasks (non Javascript) which I am happy for Google to crawl.

3. Now that my site is established, do I need to continue using/creating a Google Sitemap? What is the point?

It’s been a while I know but I thought I would post about Yzoo.co.uk

In January I posted here about the site being dropped by Google and not being listed in the SERP’s. Well things did pick up again and January matched novembers traffic and earnings - Great. February did even better, reaching more than 1,000,000 (one million) visitors and matched the earnings of December, well February beat Decembers earnings by a few dollars.

Read the rest of this entry »

… I have bought a ready made site from xyz.com but it’s not making me money, or it’s not doing well in the google SERPS, why?
I am going to be frank?

You will get a thing from those sites as they are.

WHY?

1. For a site to do well in the Google results the content needs to be fresh, unique and have ‘value’. A blog that does well is a blog that is updated daily, not once a week or once a month (hence this one not doing well).

2. The content on these sites are normally copied from elsewhere - check the sellers ‘example site’ using http://www.copyscape.com and see just how unique the site is, so if you do manage to get indexed in google the chances are you won’t get a good position in the results as google etc will punish for duplicate content etc.

3. If you have bought the site then you can be sure others have also bought the exact same site. Based on #2 above, very few of these sites will ever do well in the SERPS.

4. The subjects for these sites generally contain ‘high value’ keywords. This means that although the ads being displayed may pay more, the amount of competition from other publishers will be high and unless you have good fresh and unique content with ‘value’ as mentioned above you will never rank high in google and therefor you will get little traffic. You need good traffic to earn well on adsense.

My Suggestion would be to forget about buying a site, think of a subject which you enjoy doing/talking about and know about. Start a blog about it, update it daily, this will then grow your site and get you more pages indexed better in google, it should also improve your chances of return visitors and back links to your site.

As already said, SEO requires work. Those that beleive they can buy a site or content via sites such as ebay will struggle.

Greg

I have had adsense on many of my sites for a couple of years now, never making much more than $50/month in total and never really taking adsense seriously as a genuine source of income until I started a new site in Setember 2006.

As I am a perl developer the site was more a project, a client had asked me to mock something up for them but when I had completed it they changed their mind saying they are not going to go into that area now, so rather than scrap it I kept the site going, added adsense and modified it as and when.

In September it earned me $90, this was twice what all my other sites had ever earned me from Adsense.

By December I hit $4000 for the month. Wow this is a great source of income, more then I get from selling my scripts, so rather than do more perl work I sat back thinking well at this rate I don’t need to do much, $4k/month for doing very little apart from updating the site now and again is a decent wage.

In Early January 2007 I had a big shock, the site disappeared from Googles index, my traffic dropped and so did my adsense earnings. Back down to about $30/day. OK for adsense that isn’t bad, in fact its more than any one of my sites has ever earned from adsense in a whole month, but it was a 75% drop in adsense income.

I thought all kinds of things, has the site been banned for some reason, have all my inbound links disappeared? so I spent some time reworking the site. I liked the idea of $4k+/month for doing little.

It turns out that I was not the only one to disappear from the google index and it had little to do with my site, after a few days my site started to reappear and after about 2 weeks the site was fully indexed and at the end of the month I had earned $3k.

Last Friday adsense peaked at $330 for the day. This was no surprise as each day since the January traffic/earnings increased. Then I had a ‘Google Crash’ again - Although Google shows xx000 pages from my site are indexed, when using the ’site:’ command it only shows about 50 in the results.

This in turn has killed my traffic, I dropped from 80,000 page impressions on Friday, 80% of which where from Google to 8,000 impressions yesturday, 4% of which came from Google. Adsense fell to less than $30 yesturday

I do not know if this is a similar situation as it was in January or if this time I have upset Google some how, I don’t think that is the case as I would expect them to ask me to remove adsense from the site if it is not suitable. I do know though that I didn’t learn anything in January but I have learnt it now. Don’t put all your eggs in one Basket

Rather than ignoring other projects, I should have, and will be doing now, worked on other projects so if this site fails for whatever reason I have something else to fall back on. I have to work on this fast as a drop from $3k-$4k/month to $1k/month is a big drop for me and most people and not one that we can afford.

So my advise to anyone reading this, if you hope to earn online, spread your eggs about, do not rely one one single source of income when working online, it is a volatile business and a good earner could become a no earner over night with no warning.

Greg

In my last post I mentioned that in January Google basically dropped one of my sites from their index. Well it has happened again. I do not know what is happening nor how long it will last for.

Currently Google says there are 103,000 pages indexed from my site but once you start browsing through the Google results for my site it drops from 103,000 to 80 pages in the google index. On Friday just gone, the 103,000 was about right, 1000 of the pages where in the main index with the rest (older out of date) where in the supplimental index. Yesturday (Saturday) that dropped to just 43 pages in the main index and none in the supplimental. Today it has crept up to 80 but still a small percentage of what it should be.

I use the Google Sitemaps and that contains about 140,000 urls and has no errors but the google bot, although crawling my site, is not crawling the pages in my sitemap but crawling my javascript files.

How is this effecting my traffic? drastically to be frank. For the 7 days prior to this happening, almost 80% of my traffic orriginated from Google, yesturday (when the site was dropped) that dropped to about 60%, today it has dropped to about 3%

My traffic peaked this week at around 80,000 impressions in a day, with an average of around 55,000/day for the month of March up until Friday. Yesturday I had a mere 10,000 page impressions and today it looks like it will be about 5000 impressions. You can imagine the effect this has had on my advertising/adsense revenue.

The question is - Should I pannic?

Well in January I did, I made changes to my site thinking they would improve the situation but they had very little effect and I found that other sites had experienced the same problem. About 2 weeks later everything appeared good and the site was fully indexed in google and my adsense/advertising revenue was on the up again.

The difference this time is that I have heard very few reports of this happening elsewhere, but I don’t beleive the site has been blocked from the google SERPS as my adsense account is still active. Surely if Google doesn’t like my site for any particular reason they would have asked me to remove adsense from it?

I have decided to do nothing for the next few days, it will hurt my pocket more now than it did in January as the site is more popular but if I do make changes I could make things worse. I will update on here in the next few days to let you know the outcome.

If you are in the UK then you would have most likely heard about people reclaiming bank charges from their banks on the grounds they are unlawful.

I am in the process of doing this as I was supprised by exactly how much I had been charged, although you can claim back upto 6 years of charges, I personally am only claiming back the past 4 years as to keep the amount claimed under the