The last few months complaints about poor performance and an increasing number of 502 errors have been piling up. The later were especially puzzling as it means there is a problem in the communication between the web server and uwsgi instance in the container. While there was some load on the machine everything looked fine at the first glance and there was nothing in the logs that hinted at a bug.

At the same time there were similar complaints for the web2py application that runs on the same machine. Well, actually this thing is a VM that runs on a server of a friend of a friend that does a few other things and is only maintained to the bare necessity. When I put Boxes.py there the question I asked myself was not "How do I host the greatest box generator on the internet?" but "Now that I hacked argparse to produce web pages how can I show this to a few people?"

Over the holidays I had time to look a bit deeper. Turns out the web2py app offers the data with different sorting and filtering and a horde of web crawlers eating it all up. Adding a robots.txt got rid of them and dropped the load from 0.7 to below 0.1. I may loosening this up a bit in the future but for now I just banned bots completely - including from Boxes.py.

Looking at the logs from Boxes.py it was very apparent that there are even more connections coming in. Most of them of course are these nice pop-up images in the menu. While I still didn't have access the the main machine it dawned on me that the Boxes.py instance probably gets throttled down due to too many request. This would create timeouts and though the 502 errors. To fix that I moved all the pictures to the GH (documentation) pages and adjusted the links to point there. As this would be very annoying for people that run Boxes.py locally (me) this change affects the wsgi mode only. Yes, the boxesserver should grow a config file for things like that but I just couldn't be bothered.

So performance has notably improved and I hope you all can now create your boxes without seeing 502 error messages. Looking into the access log it turns out there are quite  a lot of you. There are tens of thousands of request per day - even with the pictures moved elsewhere. This is kinda cool but also kinda scary...