Forum Access Issues & Cloudflare Challenges

According to error logs, the site (specifically our forum) started experiencing issues at around 20:06 (Sep 15) to around 07:25 (Sep 16) EEST. This resulted in an HTTP 500 server errors or white pages when attempting to access forum content (anything in `/community/`).

What happened? It appears that we got hit with some form of automated traffic (e.g., bad bot, data scraper) that exhausted the memory allocated to our database servers. I'm still investigating the origin and why it caused just the forums to lock up, but after some digging around early this morning I noticed unusual spikes in traffic deemed "likely automated" by Cloudflare. This type of traffic is typically very high on most websites (like close to half), but during this period it greatly exceeded human traffic.

Based on this, I've decided to increase Cloudflare security levels to challenge or block any bot traffic that's not verified indefinitely. While this might catch some legitimate traffic and users, it should stop something like this from happening again and reserve resources for our users, visitors, and verified bots (e.g., Google, Bing, Yahoo).

Please note that challenges are managed by Cloudflare (not the site) and will vary based on perceived threat level scores. Most challenges on humans acting like humans should be barely noticeable or require little to no action, however, if you're presented with a challenge, and assuming you pass the challenge, you'll be able to use the site for a period of time before potentially being challenged again.

I'll continue to monitor this week and provide further updates as needed.

Thanks for your patience.

I need a beer.

 

Damn, orcas are attacking everything these days.

 

You were probably asleep while all of this happened.