Problematic visitors?

[LemonYellow]

When I had a look at the little area that shows who is currently browsing the forum I noticed this:

   

What is Facebook and Twitter doing here? Do they crawl the web the same way Google does? Is it possible to prevent them from visiting?

[monsieur]

I am not sure, I will try to search the answer on web for this

[monsieur]

I've created a group bots and added all spiders/web crawlers there, and disabled ability to view forum earlier.

[monsieur]

They shouldn't show up anymore.

[LemonYellow]

They shouldn't show up anymore.

Nice. Thank you! I suspect they are crawling the web with malicious intents. At least looking for content to be used in various not-so-ethical stuff (AI-related work, for example).

[monsieur]

They shouldn't show up anymore.

Nice. Thank you! I suspect they are crawling the web with malicious intents. At least looking for content to be used in various not-so-ethical stuff (AI-related work, for example).

Yeah, they do
I found a suggestion earlier on mybb forums, to create a group bots, and then add all spiders/crawlers to group bots, revoke all permissions to view forums and stuff, and voila.
However this option was still on

Code:

Yes, show users of this group on the member list

So I unticked that, they shouldn't show up anymore in the members list.

Let's wait if their names disappear

[monsieur]

So I disabled them from viewing the forum, but it looks like they still show up in online list...

[monsieur]

Who's Online Displays Spiders?
Display spiders in the online list. Note: This setting only takes effect on the portal and index pages.

Ok they also won't show up anymore :-).
Bye bye crawlers

[LemonYellow]

Who's Online Displays Spiders?
Display spiders in the online list. Note: This setting only takes effect on the portal and index pages.

Ok they also won't show up anymore :-).
Bye bye crawlers

Is this because they really cannot view the forum now, or because their visits are no longer displayed to us, the users. The whole thing looked a bit troubling to me. This was the first time I actually saw Facebook and Twitter visiting/crawling and possibly scraping websites. They are not search engines.

Do we have any widgets or buttons that connect to those sites? For example, "share" or "like" buttons that uses a Facebook or Twitter login? That could be counted as a way of inviting them to collect data.

But if there is no such thing in place, they may be doing something against the law in some parts of the world. Unethical altogether.

[monsieur]

Hey, I am not able to write back right now. Os on my laptop crashed and I am not able to boot, how nice.

I think I am going to get back to openBSD

there's a spider/crawlers list in the admin panel, I added all of them to the group bots and disabled the ability to view the forum. They probably still come here but can't see anything. I will post some screenshots later, I haven't got my usb stick with me.right now.

[monsieur]

That's the list

[LemonYellow]

That's the list

I see! So Facebook and Twitter seem to visit here more often than Google! What a weird thing.

[monsieur]

Yeah, it's creepy, ain't it.

Nice of mybb devs to include all these options for admins though.