BotBanish Firewall Client

[showngo]

Im on my phone now, at work. Ill get that for you in the morning.  Im running the current version of SMF and downloaded the botbanish from here two days ago from the Mods section.

With checking my server it seems i was getting bogged down at there level with ip checks. I have about 800 members on at a time but im not sure.

As in my OP, there was no major issue other than a delay accessing the forum.

[Randem]

Look in the folder where SMF is loaded. You should find a folder named logs. See if there are files in there. If so I would like to have a look at them. All that line 57 does is to do an insert into a table in your local database with ignore and that should not cause any issues. The only other issue could be DNS issues when originally connecting to BotBanish to check proper credential that causes a delay. This would be a one time check for logged on users.

What is your domain name so that I can check on the server.

BTW: IP checks are not done on users that are logged in. They are only done on guests... (not logged in) meaning someone who is attempting to log in.

[showngo]

Ill get to the file.   The domain is weaponsguild.com

[Randem]

Yes, your domain is not very active with hits on our server and your domain hasn't hit our server since 2/20/2018 so the only delay that I  could be a DNS check but not a server hit on IP checks. But if you are not hitting our server that would negate the DNS check on our server unless you are getting a timeout.

PS: BotBanishClient does not seem to be installed on your system... Can you verify?

[showngo]

Yes, your domain is not very active with hits on our server and your domain hasn't hit our server since 2/20/2018 so the only delay that I  could be a DNS check but not a server hit on IP checks. But if you are not hitting our server that would negate the DNS check on our server unless you are getting a timeout.

PS: BotBanishClient does not seem to be installed on your system... Can you verify?

No it is not, Like I posted on the 20th.  I removed mine after the error log had 127 lines after 10 hrs, I not reporting an "error" I was commenting on the poster before that , saying, after removing the package from package manager there are no errors or "lasting effects" they stated that after they removed it there forum was still encountering problems.   

[Randem]

Oh Ok good to know. If you want to re-install and need some insight, just let me know.

I could possibly help with the logon delay. What other MODs do you have installed?
Could I possibly see your .htaccess file?
Also the exact complete error from the log file for line 57?

I still would like to track it down...

[Starbuck501]

Yesterday I installed BotBanish Firewall Client 2.2.05 on SMF 2.0.15
at https://www.smokey-services.eu/forums/index.php

Today I received the following 3 message...

A BOT/USER has been terminated from accessing the system

IP Address: 42.236.49.26
User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36; 360Spider

--------

A SetEnvIfNoCase entry was not found in the .htacces file and the User-Agent is still being allowed entry to the system

IP Address: 42.236.102.157
User Agent: 360Spider

----------

A BOT/USER is attacking the system. It has been terminated

IP Address: 211.108.3.235
User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.104 Safari/537.36

----------

What is 360Spider?

Since installing BotBanish I've also noticed that I'm not seeing the following:
BingBot
Google (Bot2)
Yahoo!
Is this by design or is it just a coincidence ..... obviously I don't want to stop the search spiders from visiting.

[Shambles]

What is 360Spider?

http://www.botreports.com/user-agent/360spider.shtml

[Starbuck501]

Thanks for the reply Shambles.

Nice link.

[Randem]

Starbuck501,

Most messages and explanations are found here on the Official BotBanish Support site.

BingBot
Google (Bot2)
Yahoo!

Some spiders are not really good ones bandwidth wise and are  fake spiders spoofing to be search engines. The real spiders by name are in the good spiders detection mechanism and are not being blocked but some fake spiders are spoofing them and we stop them. You can override the BotBanish Spider Detection Mechanism if you feel this is in error to what you really want.

A BOT/USER is attacking the system. It has been terminated - BotBanish is not allowing this IP to access your system.

PS: You get a message every time that Botbanish locks out and IP or User_Agent, so if you have not gotten a message about on of the mentioned Bots; BotBanish did not block them.

[Starbuck501]

Some spiders are not really good ones bandwidth wise and are  fake spiders spoofing to be search engines. The real spiders by name are in the good spiders detection mechanism and are not being blocked but some fake spiders are spoofing them and we stop them.

So what you're saying is.... the real Google/Bing/Yahoo bots are not prevented from searching the site?
only the ones pretending to be legit are blocked.

[Randem]

That is correct. If you want to be sure, take a look at the NEW Google Search Console for your site by logging into your Google account. You should see that Google is still searching your site.

Another way is to look at your .htaccess file and see that Google, Bing nor Yahoo are being blocked in the file.

[Starbuck501]

Ok, thank you for that confirmation.

[Starbuck501]

take a look at the NEW Google Search Console for your site by logging into your Google account. You should see that Google is still searching your site.

Seems there's been nothing since installing BotBanish.

Another way is to look at your .htaccess file and see that Google, Bing nor Yahoo are being blocked in the file.

Nothing in the .htaccess file.

Looking through the 'Logs' folder though, I see entries like this:

66.249.66.6 - - [28/Feb/2018:00:10:07 +0100] "GET /forums/index.php?topic=10864.0 HTTP/1.1" 403 631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.6 - - [28/Feb/2018:00:10:13 +0100] "GET /forums/index.php?board=190.%1$d HTTP/1.1" 403 631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
40.77.167.18 - - [28/Feb/2018:00:10:15 +0100] "GET /forums/index.php?board=199.240 HTTP/1.1" 403 631 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
66.249.66.6 - - [28/Feb/2018:00:10:19 +0100] "GET /forums/index.php?board=207.%1$d HTTP/1.1" 403 631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.6 - - [28/Feb/2018:00:10:48 +0100] "GET /forums/index.php?topic=309543.0 HTTP/1.1" 403 631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.6 - - [28/Feb/2018:00:10:59 +0100] "GET /forums/index.php?topic=219207.0 HTTP/1.1" 403 631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
40.77.167.18 - - [28/Feb/2018:00:11:37 +0100] "GET /forums/index.php?topic=324944.0 HTTP/1.1" 403 631 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

Any idea as to the problem?
Like I said earlier, the search engines visited regularly before installing BotBanish.

[Randem]

I need to see your .htaccess to see if there is a bad_bot entry in there with www.
If so remove that and all should be good.

[Starbuck501]

This is the complete .htaccess file.............

RewriteEngine On
RewriteCond %{HTTPS} !=on
RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]

# PRETTYURLS MOD BEGINS
# Pretty URLs mod
# http://code.google.com/p/prettyurls/
# .htaccess file generated automatically on: February 20, 2018, 23:13

RewriteEngine on
RewriteBase /forums

# Rules for: boards
RewriteRule ^([-_!~*'()$a-zA-Z0-9]+)/?$ ./index.php?pretty;board=$1.0 [L,QSA]
RewriteRule ^([-_!~*'()$a-zA-Z0-9]+)/([0-9]*)/?$ ./index.php?pretty;board=$1.$2 [L,QSA]

# Rules for: topics
RewriteRule ^([-_!~*'()$a-zA-Z0-9]+)/([-_!~*'()$a-zA-Z0-9]+)/?$ ./index.php?pretty;board=$1;topic=$2.0 [L,QSA]
RewriteRule ^([-_!~*'()$a-zA-Z0-9]+)/([-_!~*'()$a-zA-Z0-9]+)/([0-9]*|msg[0-9]*|new)/?$ ./index.php?pretty;board=$1;topic=$2.$3 [L,QSA]

# PRETTYURLS MOD ENDS
# POST AND PM INLINE ATTACHMENTS BEGINS
RewriteEngine on
RewriteRule attachment_(\d+)\.(\d+)_(\d+)\.pdf index.php?action=dlattach;topic=$1.$2;attach=$3
# POST AND PM INLINE ATTACHMENTS ENDS

[Randem]

That is the incorrect .htaccess file. Look for the one in the root folder location, that one is from the SMF folder location.

[Randem]

The entry would look like this:

SetEnvIfNoCase User-Agent "www." bad_bot

Just remove it. Always make a backup copy first.

[Starbuck501]

Sorry about that, am still learning as I go along.

Yes I do see that entry:

SetEnvIfNoCase User-Agent "www." bad_bot
SetEnvIfNoCase User-Agent "Yandex" bad_bot
SetEnvIfNoCase User-Agent "ccbot" bad_bot
SetEnvIfNoCase User-Agent "cliqzbot" bad_bot
SetEnvIfNoCase User-Agent "seznambot" bad_bot
SetEnvIfNoCase User-Agent "bubing" bad_bot
SetEnvIfNoCase User-Agent "360Spider" bad_bot
SetEnvIfNoCase User-Agent "go-http-client" bad_bot
SetEnvIfNoCase User-Agent "mj12bot" bad_bot
SetEnvIfNoCase User-Agent "webcrawler" bad_bot
SetEnvIfNoCase User-Agent "gluten free crawler" bad_bot
SetEnvIfNoCase User-Agent "megaindex" bad_bot
SetEnvIfNoCase User-Agent "zgrab" bad_bot
SetEnvIfNoCase User-Agent "; obot" bad_bot
SetEnvIfNoCase User-Agent "semrushbot" bad_bot

------------

So it's just that entry to remove?

[Randem]

Yes