Thread: Database error.
View Single Post
Old 03-17-25 | 06:23 PM
  #103  
Duragrouch
Highly Enriched Driftium
5 Anniversary
Community Builder
Community Influencer
 
Joined: Apr 2017
Posts: 6,667
Likes: 2,156
I'm just a user, not admin.

Just a brief summary from early in this thread, for those new to it:

- This is not system-specific. Not your computer or phone, not operating system, not browser. It's not you.

- This database error is caused by the Bike Forum servers being overwhelmed by "web crawlers", which "scrape data" from websites. It's like if every BF user, or many more than that, were all simultaneously flipping through every page on every thread, in order to read every post, all at once. In years past, the motive might have been to get data for targeted advertising and other things. Now, it seems to be powered by companies/entities/countries competing in Artificial Intelligence, which is now driven by "Large Language Models", which attempt to read every single thing published on the internet, and due to the feverish competition, do it frequently to build their database as quickly as possible. These data searches are supposed to be "polite", in a way that does not crash servers. Of late, they are not. The techs at BF have tried to detect this occurance, and put in a time delay or blocking a website, but it is difficult. I am guessing that web crawlers can easily change or disguise their identity and origin, making it difficult to permanently block a particular one.

- The BF servers had been down for many hours in the past, but in recent months, seems to reboot(?) in a matter of minutes when an error occurs. Not perfect, but way better than it was.

- Permanent resolution of this problem may require (only my guess), enforceable rules/legislation that would make these crawlers self-identify, follow the rules of the road, respect refusals, etc. That's way, WAY above my knowledge and understanding of the web, not my profession. If I can remember, next time I see someone I know who is retired from high up in computer tech, and is still active in trade associations, I'll ask if they know answers/suggestions on this.

Last edited by Duragrouch; 03-17-25 at 08:32 PM.
Duragrouch is offline  
Reply