Dealing with an unresponsive server

Say you have a server that is currently under very heavy load. Thankfully you’ve already had a shell open, but the load is so high, even a quick check of the load takes 5 minutes.

This is where the & is your friend. This single character will fork the process in the background, allowing it to be run at the CPU’s convenience.

Say this load is caused by multiple HTTP requests (tens of thousands) to your server, from a few IP addresses. One can use IPTables to drop a huge chunk of the packets from those IP addresses – mitigating the risk.

The following will limit all IP addresses to ten connection limit on port 80. This is, of course, if sudo still works – often times under high load sudo’ing up will break at checking if you’re in the sudoers list. In that case use su instead.

 sudo iptables -A INPUT -p tcp --syn --dport 80 -m connlimit --connlimit-above 10 -j REJECT --reject-with tcp-reset &

Now you can stop Apache, and give yourself some breathing room.