Currently I have been running my Blog on http://baudhayan.blogspot.com , also accessible through http://tinyurl.com/gadgetguru from past 5 - 6 years. Very recently I registered myself with a Domain registration site noip.com . Since they offered few free sub-domains I registered http://gadgetguru.myvnc.com with them which now pointed towards http://baudhayan.blogspot.com . As instructed by Blogger.com I also made the necessary changes on Blogger and noip.com so that my Blog was now publicly viewable as http://gadgetguru.myvnc.com . Once the DNS settings refreshed on noip.com http://gadgetguru.myvnc.com could now be used in the browser to view my Blog.
The first hint of some Malware I realised it was when I tried posting a Status Update on Facebook with http://gadgetguru.myvnc.com . Facebook immediately popped up an error message saying that I could not use the above URL due to security restrictions. I then deleted the registration of gadgetguru.myvnc.com from noip.com and restored by original Blogger settings so that my Blog is accessible via the traditional URL
http://baudhayan.blogspot.com . Though I had done the changes the damage was already done. I started receiving some strange mails from Google Webmaster Tools with the subject-line "[Webmaster Tools] Message summary". The mail contents were as follows -
___________________________________________________________________________________
Message summary
Webmaster Tools
sent you the following important messages about sites in your account.
To keep your site healthy, we recommend regularly reviewing these
messages and addressing any critical issues.
Over
the last 24 hours, Googlebot encountered 1 errors while attempting to
access your robots.txt. To ensure that we didn't crawl any pages listed
in that file, we postponed our crawl. Your site's overall robots.txt
error rate is 100.0%.
You can see more details about these errors in
Webmaster Tools.
Recommended action
If the site error rate is 100%:
- Using a web browser, attempt to access http://avalss.myvnc.com/robots.txt.
If you are able to access it from your browser, then your site may be
configured to deny access to googlebot. Check the configuration of your
firewall and site to ensure that you are not denying access to
googlebot.
- If your robots.txt is a static page, verify that your web service has proper permissions to access the file.
-
If your robots.txt is dynamically generated, verify that the scripts
that generate the robots.txt are properly configured and have permission
to run. Check the logs for your website to see if your scripts are
failing, and if so attempt to diagnose the cause of the failure.
If the site error rate is less than 100%:
- Using Webmaster Tools,
find a day with a high error rate and examine the logs for your web
server for that day. Look for errors accessing robots.txt in the logs
for that day and fix the causes of those errors.
- The most
likely explanation is that your site is overloaded. Contact your hosting
provider and discuss reconfiguring your web server or adding more
resources to your website.
- If your site redirects to another
hostname, another possible explanation is that a URL on your site is
redirecting to a hostname whose serving of its robots.txt file is
exhibiting one or more of these issues.
After you think you've fixed the problem, use
Fetch as Google to fetch
http://avalss.myvnc.com/robots.txt to verify that Googlebot can properly access your site.
Learn more in our Help Center.
Over
the last 24 hours, Googlebot encountered 2 errors while attempting to
access your robots.txt. To ensure that we didn't crawl any pages listed
in that file, we postponed our crawl. Your site's overall robots.txt
error rate is 100.0%.
You can see more details about these errors in
Webmaster Tools.
___________________________________________________________________________________
The above content is just an extract of a large mail where Google talks about several sites which the Google Bot is unable to crawl while indexing. I was surprised that none of these sites were mine, but I was receiving the error messages. I logged on to the Google Wemaster Tools page and tried searching a lot but couldn't find any references to these sites.
After searching the Google forums I found a solution - visit https://www.google.com/webmasters/verification/home?hl=en&authuser=0 and look into the list of sites where you appear as the Verified Owner. Begin with the 1st site on which you are receiving the error. In my case I just saw 2 sites which were http://baudhayan.blogspot.com and http://sites.google.com/site/baudhayan . I did not find any reference to http://avalss.myvnc.com . But as instructed in the forum I clicked on "ADD A SITE" and added http://avalss.myvnc.com to the list. To my surprise it added the site and showed me as the verified owner. On clicking the site it also showed various other website ending with myvnc.com where I was shown as the Verified Owner since myvnc.com somehow poisoned the DNS records and showed be as the verified owner. I quickly clicked on "Unverify" and deleted http://avalss.myvnc.com from the list of my sites.
Though I do not know the entire details about how this Malware is introduced but my understanding is that the DNS registration causes the Google Bot to start crawling across the entire domain myvnc.com which has a suspicious robots.txt which requests Google to crawl through many more sub-domains on myvnc.com and each of these domains report me as the Verifies Owner.
It looks like I have fixed the problem as of now but would like to warn my readers about doing some research on the Net before you use any free sub-domain and plan to re-point your URL.
