All my Siteground sites went offline for a few hours on Saturday due to limits on the volume of traffic being broken, what Sitegound refer to as the number of scripts being run. In my case the limit is 40k per day.

I rapidly put all the sites through Cloudflare but the number of hits is still excessive even now. One site in particular is being hit excessively nikamusicaltheatre and from the Cloudflare analytics page I can see that the problem is the Yandex bot which is a Russian search engine bot. Its hitting this site 26k times in 24 hours. None of my other sites are being crawled like this one. 

Searching the web I came up with this solution in the htaccess file to bar Yandex:


## Bar Yandex bot ##
SetEnvIfNoCase User-Agent "^Yandex*" bad_bot
Order Deny,Allow
Deny from env=bad_bot

and also this one:

SetEnvIfNoCase User-agent “Baidu” spammer=yes
SetEnvIfNoCase User-agent “Yandex” spammer=yes
SetEnvIfNoCase User-agent “Sosospider” spammer=yes

<Limit GET PUT POST>

order deny,allow

deny from env=spammer
</Limit>


22nd August Update

Above solutions seem not to work. Now up to 46k hits per 24hrs.

Now trying this code instead:

SetEnvIfNoCase User-agent (yandex|baidu|foobar) not-allowed=1

 Order Allow,Deny
Allow from ALL
Deny from env=not-allowed

And also blocking by IP address all Russian IP addresses

and this in robots.txt file:

User-agent: Yandex
Disallow: / # blocks access to whole site

23rd August 2017

Well that worked. Siteground stats back to ground level.

Cloudflare stats show that Yandex is still hitting the site and that Russia has 65k hits so thats got me wondering about the relationship between what Cloudflare sees and the hosting but at least its sorted the immediate bandwidth problem at Siteground.