Blame
Date:
Mon Jan 23 05:00:25 2023 UTC
Message:
Daily backup
01
2023-01-22
jrmu
version=pmwiki-2.2.130 ordered=1 urlencoded=1
02
2023-01-22
jrmu
agent=w3m/0.5.3+git20210102
03
2023-01-22
jrmu
author=jrmu
04
2023-01-22
jrmu
charset=UTF-8
05
2023-01-22
jrmu
csum=
06
2023-01-22
jrmu
ctime=1648138925
07
2023-01-22
jrmu
host=38.87.162.154
08
2023-01-22
jrmu
name=Ddos.Intro
09
2023-01-22
jrmu
rev=1
10
2023-01-22
jrmu
targets=Openbsd.SSDP,Openbsd.ACKFlood,Openbsd.Tcpresetflood,Openbsd.Cu,Openbsd.Vmmuser,Openbsd.Buyvm,Openbsd.Tcpdump,Openbsd.Scp,Openbsd.Sftp,Openbsd.SYNFlood,Openbsd.HTTPFlood,Openbsd.NTPAmplification,Openbsd.UDPFlood,Openbsd.RSTFlood,Openbsd.Police
11
2023-01-22
jrmu
text=(:title DDoS Defense Guide:)%0a%0a!! How to Detect an Attack%0a%0aIf suddenly you see many users disconnect from a server...%0a%0a[@%0a[12:31:23] *** Quits: Lucifer_|des (JohnReb@AEJva.DesireNET.Org) (Ping timeout)%0a[12:31:51] *** Quits: depeche|nat (depeche@depeche.users.nationchat.org) (Ping timeout)%0a[12:32:36] *** Quits: iulian7502|des (iulian@ADx0-.DesireNET.Org) (Ping timeout)%0a[12:34:07] *** Quits: Counter|under (CPT@CPT.fig.ircnow.org) (Ping timeout)%0a[12:34:12] *** Quits: katrok|quake (katrok@katrok.bnc1.ircnow.org) (Ping timeout)%0a[12:34:19] *** Quits: Lucifer|des (Lucifer@ACXJz.DesireNET.Org) (Ping timeout)%0a[12:35:29] *** Quits: depeche|quake (depeche@depeche.users.quakenet.org) (Ping timeout)%0a[12:35:38] *** Quits: Elafi|under (Elafi@Elafi.fig.ircnow.org) (Ping timeout)%0a[12:35:52] *** Quits: edu|dal (ed@fig.ircnow.org) (Ping timeout)%0a[12:36:45] *** Quits: Soportes (Soportes@Soportes.fig.ircnow.org) ("IRCNow and Forever!")%0a@]%0a%0a...your server may be under attack!%0a%0a!! Different Attack Types%0a%0aIf you see ping timeouts like above, your server's bandwidth is clogged with so many junk packets that it cannot respond to real traffic. This could be the result of a [[openbsd/SSDP|SSDP attack]] or a [[openbsd/ACKFlood|TCP ack flood]]. Logging and analyzing incoming packets is how we identify the attack type.%0a%0aIf you see many @@EOF from client@@, @@Read error: Input/output error@@, or @@Client closed connection@@ quit messages, this may be due to a [[openbsd/tcpresetflood|TCP reset flood]]:%0a%0a[@%0a[02:02:42] *** Quits: jrmu|dal (jrmu@jrmu.plum.ircnow.org) (Read error: Input/output error)%0a[02:02:42] *** Quits: semut_|dal (semut@semut.lu2.ircnow.org) (Client closed connection)%0a[02:02:42] *** Quits: starr|dal (starr@starr.lu2.ircnow.org) (Read error: Input/output error)%0a[02:02:42] *** Quits: Gisa|dal (thekingofb@thekingofbandit.lu2.ircnow.org) (Read error: Input/output error)%0a[02:02:42] *** Quits: Gisa|quake (thekingofb@lu2.ircnow.org) (EOF from client)%0a[02:02:42] *** Quits: Freak|quake (Freak@Freak.lu2.ircnow.org) (EOF from client)%0a[02:02:42] *** Quits: IRCuser|quake (unknwon@IRCuser.users.quakenet.org) (EOF from client)%0a[02:02:42] *** Quits: ramadi|quake (ramadi@ramadi.lu2.ircnow.org) (EOF from client)%0a[02:02:43] *** Quits: Fat1 (Fatfem@Fatfem.lu2.ircnow.org) (Client closed connection)%0a[02:02:43] *** Quits: Freak (Freak@Freak.lu2.ircnow.org) (Client closed connection)%0a@]%0a%0a!! Confirming an Attack%0a%0aTo see if there is a ddos attack, first run:%0a%0a[@%0a$ ping fruit.ircnow.org%0a@]%0a%0aReplace fruit.ircnow.org with your actual server's hostname. If you get more than 30%25 packet loss, this could be a sign you are being flooded with fake packets. During a DDoS, an attacker is flooding your internet pipe with junk packets so that your server is unable to respond to any real network traffic.%0a%0aCheck the serial console (using [[openbsd/cu|cu]] if on [[openbsd/vmmuser|VMM]], VNC if on [[openbsd/buyvm|BuyVM]]). Try to log in and see if the system is still responsive. If the server is offline or has errors, it may be due to hardware/software issues rather than a DDoS attack.%0a%0aDuring a DDoS attack, your system should feel slightly sluggish. This is due to the massive number of packets that OpenBSD must process. Run @@top@@. If you see the system/interrupt CPU usage at 50%25 or more, then you either have a process using too much CPU or you may be under an attack. If the CPU usage is low, under 5%25, it might not be a DDoS attack. It could be a networking or hardware issue, or an attack of another nature.%0a%0aIf you suspect an attack, you should log the packets that are coming in:%0a%0a[@%0a$ doas tcpdump -w YYYYMMDDSS.pcap%0a@]%0a%0aThis will log the packets onto the file @@YYYYMMDDSS.pcap@@, where YYYY is the year, MM is the month, DD is the day, and SS is the serial number.%0a%0aYou can fine-tune this:%0a%0a[@%0a$ doas tcpdump -i vio0 -s 1500 -A -v -n -w YYYYMMDDSS.pcap%0a@]%0a%0aIn the above command, the argument -i specifies the interface @@vio0@@, -s 1500 analyzes the first 1500 bytes, -A prints each packet in ASCII, -v provides more verbose output, and -n avoids address conversion. Make sure to read up on [[openbsd/tcpdump|tcpdump]].%0a%0aTo stop collection, type [ctrl]+c, @@^C@@. If you log for a few seconds and only receive a few hundred packets, perhaps you are '''not''' under attack. If, however, you see millions of packets arriving within a few seconds, you are certainly under attack. Save the pcap file because it is very useful for forensics. We will use it for reporting the attacker and for blocking his attacks.%0a%0aTo read the pcap, you can transfer the pcap to your desktop using [[openbsd/scp|scp]] or [[openbsd/sftp|sftp]]. Or, more quickly, you can analyze it on the server itself using tcpdump. Use the same options but replace -w with -r:%0a%0a[@%0a$ doas tcpdump -r YYYYMMDDSS.pcap%0a@]%0a%0aor%0a%0a[@%0a$ doas tcpdump -i vio0 -s 1500 -A -v -n -r YYYYMMDDSS.pcap%0a@]%0a%0a!! Identifying Attack Type%0a%0a|| border=1 width=100%25 class="sortable simpletable"%0a||! Common DDoS Attacks !||%0a||! Name ||! Packet Type ||! Description ||%0a|| [[openbsd/SYNFlood|SYN Flood]] || SYN packets || Your server sends ACK packets and wastes resources on useless TCP connections ||%0a|| [[openbsd/HTTPFlood|HTTP Flood]] || HTTP packets || Your server wastes resources responding to fake HTTP requests ||%0a|| [[openbsd/NTPAmplification|NTP Amplification]] || NTP packets || Your server is flooded with fake NTP packets ||%0a|| [[openbsd/UDPFlood|UDP Flood]] || UDP packets || Your server receives lots of fake UDP packets at random ports ||%0a|| [[openbsd/SSDP|SSDP Attacks]] || SSDP packets || Your server is flooded with packets from Universal Plug and Play (UPnP) devices ||%0a|| [[openbsd/ACKFlood|TCP ACK Flood]] || ACK packets || Your server is flooded with useless ACK packets ||%0a|| [[openbsd/RSTFlood|TCP RST Flood]] || RST packets || Your system daemons are tricked into disconnecting by fake RST (reset) packets ||%0a%0a!! Reporting Criminals%0a%0aTo stop abuse, it is necessary to report attackers. Often, a DDoSer can be quickly identified because an attack took place during an argument. A DDoSer may also be a competitor or have some other financial motive. If the pcap shows a specific IPv6 address was targeted, contact the user with that unique IPv6 address and ask him for clues. Attackers sometimes will show up on your channel to insult or mock you during an attack. This type of evidence, although not 100%25 certain, can provide clues as to who is behind an attack.%0a%0aHints provide a very useful starting point, but you will still need to do the necessary investigation in order to get enough evidence to report. Please see the [[openbsd/police|police guide]].%0a%0aKeep in mind that criminals often spoof IP addresses (lie about the source IP address) or use proxies (other insecure machines) to amplify their attacks. Logs of packets alone often do not provide enough evidence to report a criminal. Attackers often amplify their attacks using IoT devices, universities, and large corporations to mask their true origin. The attacking IP might be the stolen server, home computer, or electronic device of some innocent bystander.%0a%0a!! DDoS Filtering%0a%0aIf you are hosting a public service, sooner or later you will get hit with DDoS attacks. The Internet is full of criminals, and the anonymity of IRC makes it hard to catch them. Strong and determined attackers can sustain DDoS attacks of more than 500Gbps.%0a%0aA software firewall on a single server alone cannot stop all DDoS attacks because of physical limitations. Once the maximum bandwidth of your server is reached, it is unable to accept any further traffic. If you get attacked but you are not using a filtered IP, your provider will null route your IP (take it offline). So long as the attack continues, your service will be completely offline, and your users will blame you.%0a%0aAs a result, we are forced to purchase DDoS filtering or acquire more bandwidth. Both solutions cost money, which is why it is necessary to work as a team to purchase more bandwidth at a cheaper rate. The larger our network, the more money we can afford to spend on DDoS defenses. It is wise to avoid depending entirely upon one single internet provider like Cloudflare or Voxility. Using diverse providers decreases the risk of failure.%0a%0a!! Troubleshooting%0a%0aCloudflare/Voxility's DDoS protection can mangle SSL certs when you make requests from some servers. For example, I noticed HTTPS requests to Let's Encrypt's servers were being mangled: %0a%0a[@%0a$ openssl s_client -connect 172.65.32.248:443%0a@]%0a%0aIt just hung there with no certificate being issued. If this happens, you will need to report this to the Internet Provider ([[https://buyvm.net|BuyVM]]).%0a%0aDDoS filtered IPs have been known to mangle SMTP, NTP, and DNS packets. So if networking fails for some inexplicable reason, check using openssl to see if SSL certs are being mangled.%0a
12
2023-01-22
jrmu
time=1648138925
13
2023-01-22
jrmu
title=DDoS Defense Guide
14
2023-01-22
jrmu
author:1648138925=jrmu
15
2023-01-22
jrmu
diff:1648138925:1648138925:=1,124d0%0a%3c (:title DDoS Defense Guide:)%0a%3c %0a%3c !! How to Detect an Attack%0a%3c %0a%3c If suddenly you see many users disconnect from a server...%0a%3c %0a%3c [@%0a%3c [12:31:23] *** Quits: Lucifer_|des (JohnReb@AEJva.DesireNET.Org) (Ping timeout)%0a%3c [12:31:51] *** Quits: depeche|nat (depeche@depeche.users.nationchat.org) (Ping timeout)%0a%3c [12:32:36] *** Quits: iulian7502|des (iulian@ADx0-.DesireNET.Org) (Ping timeout)%0a%3c [12:34:07] *** Quits: Counter|under (CPT@CPT.fig.ircnow.org) (Ping timeout)%0a%3c [12:34:12] *** Quits: katrok|quake (katrok@katrok.bnc1.ircnow.org) (Ping timeout)%0a%3c [12:34:19] *** Quits: Lucifer|des (Lucifer@ACXJz.DesireNET.Org) (Ping timeout)%0a%3c [12:35:29] *** Quits: depeche|quake (depeche@depeche.users.quakenet.org) (Ping timeout)%0a%3c [12:35:38] *** Quits: Elafi|under (Elafi@Elafi.fig.ircnow.org) (Ping timeout)%0a%3c [12:35:52] *** Quits: edu|dal (ed@fig.ircnow.org) (Ping timeout)%0a%3c [12:36:45] *** Quits: Soportes (Soportes@Soportes.fig.ircnow.org) ("IRCNow and Forever!")%0a%3c @]%0a%3c %0a%3c ...your server may be under attack!%0a%3c %0a%3c !! Different Attack Types%0a%3c %0a%3c If you see ping timeouts like above, your server's bandwidth is clogged with so many junk packets that it cannot respond to real traffic. This could be the result of a [[openbsd/SSDP|SSDP attack]] or a [[openbsd/ACKFlood|TCP ack flood]]. Logging and analyzing incoming packets is how we identify the attack type.%0a%3c %0a%3c If you see many @@EOF from client@@, @@Read error: Input/output error@@, or @@Client closed connection@@ quit messages, this may be due to a [[openbsd/tcpresetflood|TCP reset flood]]:%0a%3c %0a%3c [@%0a%3c [02:02:42] *** Quits: jrmu|dal (jrmu@jrmu.plum.ircnow.org) (Read error: Input/output error)%0a%3c [02:02:42] *** Quits: semut_|dal (semut@semut.lu2.ircnow.org) (Client closed connection)%0a%3c [02:02:42] *** Quits: starr|dal (starr@starr.lu2.ircnow.org) (Read error: Input/output error)%0a%3c [02:02:42] *** Quits: Gisa|dal (thekingofb@thekingofbandit.lu2.ircnow.org) (Read error: Input/output error)%0a%3c [02:02:42] *** Quits: Gisa|quake (thekingofb@lu2.ircnow.org) (EOF from client)%0a%3c [02:02:42] *** Quits: Freak|quake (Freak@Freak.lu2.ircnow.org) (EOF from client)%0a%3c [02:02:42] *** Quits: IRCuser|quake (unknwon@IRCuser.users.quakenet.org) (EOF from client)%0a%3c [02:02:42] *** Quits: ramadi|quake (ramadi@ramadi.lu2.ircnow.org) (EOF from client)%0a%3c [02:02:43] *** Quits: Fat1 (Fatfem@Fatfem.lu2.ircnow.org) (Client closed connection)%0a%3c [02:02:43] *** Quits: Freak (Freak@Freak.lu2.ircnow.org) (Client closed connection)%0a%3c @]%0a%3c %0a%3c !! Confirming an Attack%0a%3c %0a%3c To see if there is a ddos attack, first run:%0a%3c %0a%3c [@%0a%3c $ ping fruit.ircnow.org%0a%3c @]%0a%3c %0a%3c Replace fruit.ircnow.org with your actual server's hostname. If you get more than 30%25 packet loss, this could be a sign you are being flooded with fake packets. During a DDoS, an attacker is flooding your internet pipe with junk packets so that your server is unable to respond to any real network traffic.%0a%3c %0a%3c Check the serial console (using [[openbsd/cu|cu]] if on [[openbsd/vmmuser|VMM]], VNC if on [[openbsd/buyvm|BuyVM]]). Try to log in and see if the system is still responsive. If the server is offline or has errors, it may be due to hardware/software issues rather than a DDoS attack.%0a%3c %0a%3c During a DDoS attack, your system should feel slightly sluggish. This is due to the massive number of packets that OpenBSD must process. Run @@top@@. If you see the system/interrupt CPU usage at 50%25 or more, then you either have a process using too much CPU or you may be under an attack. If the CPU usage is low, under 5%25, it might not be a DDoS attack. It could be a networking or hardware issue, or an attack of another nature.%0a%3c %0a%3c If you suspect an attack, you should log the packets that are coming in:%0a%3c %0a%3c [@%0a%3c $ doas tcpdump -w YYYYMMDDSS.pcap%0a%3c @]%0a%3c %0a%3c This will log the packets onto the file @@YYYYMMDDSS.pcap@@, where YYYY is the year, MM is the month, DD is the day, and SS is the serial number.%0a%3c %0a%3c You can fine-tune this:%0a%3c %0a%3c [@%0a%3c $ doas tcpdump -i vio0 -s 1500 -A -v -n -w YYYYMMDDSS.pcap%0a%3c @]%0a%3c %0a%3c In the above command, the argument -i specifies the interface @@vio0@@, -s 1500 analyzes the first 1500 bytes, -A prints each packet in ASCII, -v provides more verbose output, and -n avoids address conversion. Make sure to read up on [[openbsd/tcpdump|tcpdump]].%0a%3c %0a%3c To stop collection, type [ctrl]+c, @@^C@@. If you log for a few seconds and only receive a few hundred packets, perhaps you are '''not''' under attack. If, however, you see millions of packets arriving within a few seconds, you are certainly under attack. Save the pcap file because it is very useful for forensics. We will use it for reporting the attacker and for blocking his attacks.%0a%3c %0a%3c To read the pcap, you can transfer the pcap to your desktop using [[openbsd/scp|scp]] or [[openbsd/sftp|sftp]]. Or, more quickly, you can analyze it on the server itself using tcpdump. Use the same options but replace -w with -r:%0a%3c %0a%3c [@%0a%3c $ doas tcpdump -r YYYYMMDDSS.pcap%0a%3c @]%0a%3c %0a%3c or%0a%3c %0a%3c [@%0a%3c $ doas tcpdump -i vio0 -s 1500 -A -v -n -r YYYYMMDDSS.pcap%0a%3c @]%0a%3c %0a%3c !! Identifying Attack Type%0a%3c %0a%3c || border=1 width=100%25 class="sortable simpletable"%0a%3c ||! Common DDoS Attacks !||%0a%3c ||! Name ||! Packet Type ||! Description ||%0a%3c || [[openbsd/SYNFlood|SYN Flood]] || SYN packets || Your server sends ACK packets and wastes resources on useless TCP connections ||%0a%3c || [[openbsd/HTTPFlood|HTTP Flood]] || HTTP packets || Your server wastes resources responding to fake HTTP requests ||%0a%3c || [[openbsd/NTPAmplification|NTP Amplification]] || NTP packets || Your server is flooded with fake NTP packets ||%0a%3c || [[openbsd/UDPFlood|UDP Flood]] || UDP packets || Your server receives lots of fake UDP packets at random ports ||%0a%3c || [[openbsd/SSDP|SSDP Attacks]] || SSDP packets || Your server is flooded with packets from Universal Plug and Play (UPnP) devices ||%0a%3c || [[openbsd/ACKFlood|TCP ACK Flood]] || ACK packets || Your server is flooded with useless ACK packets ||%0a%3c || [[openbsd/RSTFlood|TCP RST Flood]] || RST packets || Your system daemons are tricked into disconnecting by fake RST (reset) packets ||%0a%3c %0a%3c !! Reporting Criminals%0a%3c %0a%3c To stop abuse, it is necessary to report attackers. Often, a DDoSer can be quickly identified because an attack took place during an argument. A DDoSer may also be a competitor or have some other financial motive. If the pcap shows a specific IPv6 address was targeted, contact the user with that unique IPv6 address and ask him for clues. Attackers sometimes will show up on your channel to insult or mock you during an attack. This type of evidence, although not 100%25 certain, can provide clues as to who is behind an attack.%0a%3c %0a%3c Hints provide a very useful starting point, but you will still need to do the necessary investigation in order to get enough evidence to report. Please see the [[openbsd/police|police guide]].%0a%3c %0a%3c Keep in mind that criminals often spoof IP addresses (lie about the source IP address) or use proxies (other insecure machines) to amplify their attacks. Logs of packets alone often do not provide enough evidence to report a criminal. Attackers often amplify their attacks using IoT devices, universities, and large corporations to mask their true origin. The attacking IP might be the stolen server, home computer, or electronic device of some innocent bystander.%0a%3c %0a%3c !! DDoS Filtering%0a%3c %0a%3c If you are hosting a public service, sooner or later you will get hit with DDoS attacks. The Internet is full of criminals, and the anonymity of IRC makes it hard to catch them. Strong and determined attackers can sustain DDoS attacks of more than 500Gbps.%0a%3c %0a%3c A software firewall on a single server alone cannot stop all DDoS attacks because of physical limitations. Once the maximum bandwidth of your server is reached, it is unable to accept any further traffic. If you get attacked but you are not using a filtered IP, your provider will null route your IP (take it offline). So long as the attack continues, your service will be completely offline, and your users will blame you.%0a%3c %0a%3c As a result, we are forced to purchase DDoS filtering or acquire more bandwidth. Both solutions cost money, which is why it is necessary to work as a team to purchase more bandwidth at a cheaper rate. The larger our network, the more money we can afford to spend on DDoS defenses. It is wise to avoid depending entirely upon one single internet provider like Cloudflare or Voxility. Using diverse providers decreases the risk of failure.%0a%3c %0a%3c !! Troubleshooting%0a%3c %0a%3c Cloudflare/Voxility's DDoS protection can mangle SSL certs when you make requests from some servers. For example, I noticed HTTPS requests to Let's Encrypt's servers were being mangled: %0a%3c %0a%3c [@%0a%3c $ openssl s_client -connect 172.65.32.248:443%0a%3c @]%0a%3c %0a%3c It just hung there with no certificate being issued. If this happens, you will need to report this to the Internet Provider ([[https://buyvm.net|BuyVM]]).%0a%3c %0a%3c DDoS filtered IPs have been known to mangle SMTP, NTP, and DNS packets. So if networking fails for some inexplicable reason, check using openssl to see if SSL certs are being mangled.%0a
16
2023-01-22
jrmu
host:1648138925=38.87.162.154
IRCNow