In recent days, the evening I meet major difficulties of internet connection. Indeed I have pings to 6000ms with 50% packet loss. I noticed that my @home server off has allowed to solve the problem, which is a little strange because I imagine how one can bring to its knees a 100Mbps connection. More strange again, disable nginx and postfix services allowed also to solve problems. However the reading of these services and the system logs indicated nothing abnormal.
I me orientais therefore either to a ddos either lxc bug because the version on Debian is obsolete and one can very well imagine problems with the bridge leading to broadcast storms or whatnot of the kind. Desperate, I have upgrade my server in Jessie. Exit the kernel 3.2 and Hello to the 3.16, exit lxc 0.8 and Hello lxc1.0. Following this I noticed a return to normal and even increased the responsiveness of the server. I therefore thought the issue resolved.
But tonight, rebelotte. Ping-6000ms, 50% packet loss, and once again cutting my server was the problem disappear. This time so I did turn analysis wireshark on the network interface of the server.
The traffic with wireshark capture
# apt – get install tcpdump
On the side of laptop
Install Wireshark and then run the following commands:
# ssh firstname.lastname@example.org “tcpdump s 0-u-n w –i eth0 not port 22” >/tmp/wshark
Then, in another tab:
# wireshark k-i/tmp/wshark
Wireshark should open and start to display the traffic on the server:
Analysis of the results
So, I have observed that for two IP, the subsequent frames flooded the server:
3 0.023193000 18.104.22.168 192.168.0.2 TCP 66 [TCP Port numbers reused] 80→25 [SYN] Seq = 0 Win = 8192 Len = 0 MSS = 1460 WS = 256 SACK_PERM = 1
Without knowing all the details it seems to be so much an attack. Apparently this is the SYN flood.
Using iptables, I have blocked the two IPs in question:
# iptables FORWARD s 22.214.171.124-I j DROP
# iptables FORWARD s 126.96.36.199-I j DROP
Note: I use FORWARD because I add this rule on the host while the packages are destined for an LXC container which has a different IP address. INPUT must be put in place if this is not the case, or in doubt, both.
Miracle, after having blocked these two IP, my internet connection is returned to normal.
My second action was to run a whois on these two IP to learn a little more. They belong to OVH dedicated servers, I have therefore completed form abuse on their site to notify them that these servers have a strange behavior.
My interpretation is as follows: two servers me were inundated with requests arriving on nginx and postfix since they are destined for ports 25/80/443 according to Wireshark. If my container LXC is off, they end up in a vacuum. On the other hand if it is lit, they lead and multiply up to saturate the BBox (the router upstream), causing slowdowns. Wireshark helped me a lot to diagnose this problem, it is now my best friend.
iptables has instructions to “drop” (DROP) packets from these two addresses, which therefore amounts to reproduce the behavior of the server off. Reporting to OVH will allow, I hope, to cut these two probably compromised servers.
In 4 years of self-hosting this is the first time that I am faced with such a problem. It goes back to the reasons which led me to outsource blog maniatux.fr on a VPS OVH, namely that if attacks should be held, as well as this does itself not in me. But still me have a @home server I thought enough “discreet” for not being attacked. I hope that this will not happen in the future…