Forum guest count from China goes way up

[EL34]

Just lately when I view Who's Online at the bottom of the forum, I noticed a much larger number of guest.
When I look at the IP's I see 90% of them coming from China

Guest count has been typically less than 10

Not sure what is going on here?
For example, 105 guest on the forum and 3 members.

They can't do anything to the forum, so it is harmless I suppose.
I have good spam control for new member checks and only admin approval.

Is anyone else having this same experience on their forum?
Anyone have a clue why this just started happening?

here's an example of a IP lookup

[a10]

:O)

https://www.simplemachines.org/community/index.php?msg=4176431

[EL34]

:O)

https://www.simplemachines.org/community/index.php?msg=4176431

Thanks, but that post is not what is happening to my forum or my private cloud server.
No one is running php scripts

[Kindred]

They are bots

[EL34]

I have a bunch of listings in my robots.txt file
Are there new bots I should be blocking that are not on my list below?

Code Select Expand

User-agent: Baiduspider
User-agent: Baiduspider-video
User-agent: Baiduspider-image
Disallow: /
User-agent: Bytespider
Disallow: /
User-agent: Sogou web spider
Disallow: /
User-agent: Sogou inst spider
Disallow: /
User-agent: Yandex
Disallow: /
User-agent: magpie-crawler
Disallow: /
user-agent: AhrefsBot
disallow: /
User-Agent: trendictionbot
Disallow: /
User-agent: WeSEE
Disallow: /
User-agent: proximic
Disallow: /
User-agent: CCBot
Disallow: /
User-agent: ZumBot
Disallow: /
User-agent: DotBot
Disallow: /
User-agent: Seznam
Disallow: /
User-agent: MJ12bot
Disallow: /
User-agent: WeSEE:Ads/PageBot
Disallow: /
User-agent: MojeekBot
Disallow: /
User-agent: NerdyBot
Disallow: /
User-agent: OrangeBot
Disallow: /
User-agent: spbot
Disallow: /
User-agent: Applebot
Disallow: /


[Kindred]

Robots.txt only works if they pay attention to it. Most AI and spam bots ignore it.

You need to block them using .htaccess  --  like the above linked list says.

Also, please use code bbc tags

[a10]

General comment about the htaccess deny's link above, it's dangerous to have such an overkill number of entries. Sooner or later one ends up blocking legit visitors, either by personal error or ip ranges gets moved to other countries etc.

ID'ing those chinese bots was done more as hobby then as them posing any practical threat to the forum. Just scraping bots, seemed they wanted to download anything and everything, over and over, week after week, feeding some stupid artificial intelligence stuff?

Even with daily pageviews jumping from normal around 5000 to 50.000++ there was no sign of slowdowns nor reactions from host, so no real safety reason for the htacess, just an evening of "ip-hobby" + some satisfaction in serving them 403's instead of content (not that they care).

[EL34]

My cloud server is a Windows server

So far, I have not seen any impact on my server.
Was mainly curious if anyone else has seen a dramatic rise in Chinese IP's

I have a way to block specific IP's using MADgui on my server
I do that once in a while when I see people abusing .php scripts

[@rjen]

Have seen it years ago, and found that there was no purpose allowing them on my site, so I use GEOIP in .htaccess to keep them all out..

put this in your .htaccess file ...

Code Select Expand

# Countryfilter to keep the buggers out
GeoIPEnable On     
SetEnvIf GEOIP_COUNTRY_CODE CN DenyCountry 
SetEnvIf GEOIP_COUNTRY_CODE_V6 CN DenyCountry
order allow,deny
deny from env=DenyCountry
allow from all

[Antechinus]

Nice idea. I'm stealing that one.

[a10]

Have seen it years ago, and found that there was no purpose allowing them on my site, so I use GEOIP in .htaccess to keep them all out..

The china bots finally disapeared here, all peaceful. Next time will definitely try that tip. Getting rid of .cn (& f.ex. .ru) with just a few lines would be great. Is it a standard htaccess code, or does it need host to have some geolookup setting enabled ?

[@rjen]

Geoip must be supported by your host, but I think most hosts support it by default. Just try it

[a10]

^^^ tested geoip, 500 Internal Server Error. May chat with host someday to see if they could\would configure it to work, a very nice tool to have available.

[bayonetbrant]

Have seen it years ago, and found that there was no purpose allowing them on my site, so I use GEOIP in .htaccess to keep them all out..

put this in your .htaccess file ...

(SNIP)

Where exactly does this get pasted into the .htaccess file ?

[EL34]

I am wondering now if I am getting paid for the bot scrapings?

My forum has a google banner ad at the top of every page
You can defeat it with an ad block extension in your browser.

But, am I getting credit for the bots views?

I did notice that my google ad revenue went up the last couple months.
I look and see over 100 guest sometimes with only a few members, so the bots are busy.
Looking at the IP's. they are definitely from China

[Steve]

Are you're final questions still valid?

[Aleksi "Lex" Kilpinen]

I am wondering now if I am getting paid for the bot scrapings?

This is a difficult question - You shouldn't really, but on the other hand it's not impossible that some of the bots are counted as valid users, and that some of those would end up counting towards your ad revenue. I would even go as far to say that it is likely, that a small portion of your income is actually bots.

[a10]

Now it's Amazon (+ some chinese), moving towards 100.000+ hits in a day. Have put 3.*.*.*  & 18.*.*.* temporarily in ban list, to monitor activity. Goes into htaccess if continuing.

[shawnb61]

Yep, everything has been in overdrive the last few days...

Mostly Chinese bots, but some of this traffic is from ISPs in Italy & the UK.

Also...  "FastPlanet" from Dallas, TX, has joined the fray, with activity identical to that of the Chinese bots.

[vbgamer45]

Going to force a lot of people to move to cloudflare like soltuions.