#38598 [BC-Insight] GetReceiptsMsg abuse leads to the DoS and/or crash of every EL client in the Ethereum network
Submitted on Jan 7th 2025 at 13:56:46 UTC by @`redacted user` or Attackathon | Ethereum Protocol
Report ID: #38598
Report Type: Blockchain/DLT
Report severity: Insight
Target: https://github.com/ethereum/go-ethereum
Impacts:
Network not being able to confirm new transactions (total network shutdown)
Unintended chain split affecting greater than or equal to 25% of the network (Network partition)
All EL clients vulnerable
Description
Preamble
This report has been marked as High as it can be adjusted, but these would be classified as Critical elsewhere in the industry. I'm not sure why that hasn't been there case here. I've found ~20 exploits in various Ethereum CL/ES clients that haven't been submitted. I'll sweep this contest by the rules laid out in the terms of engagement if you let me. There are just so many to report and I don't know if there's enough time.
For practicality's sake we could establish TG DM or group comms for TL;DR reports and quick patches to circumvent time constraints and the (I don't know why) necessity of massive organically written reports on Immunefi.
This is probably a total network shutdown, or such a significant partial-network shutdown that could theoretically lead to hard fork or an easy 51% attack. One technique to double spend is to send a tx to an exchange, cash out, roll back the state of the blockchain, send it again, etc. and this is made easy when the network is barely if at all talking to itself.
I already kicked the prize pool from 250 to 500 but it should've been 900 to 1.5 based on the Attackathon's rules that follow Immunefi's v2.3 classification guidelines. Looking to bump it higher with this report.
I'm prepared to surgically dismantle any technical argument about this attack and its efficiency if necessary, but considering reward payouts are guaranteed and funded - I don't anticipate needing to social engineer anyone into fairness.
Security exploits are like math. It's either 1 + 1 = 2 or bullsh*t. I've provided functioning attack code for Geth, Nethermind and Besu in this report (albeit independently and not in a unified attack script) below. They work against synchronized clients.
Use them.
Intro
Multiple remote OOM crash, > 100% machine CPU and MEM DoS vulnerabilities have been identified in the latest releases of Geth, Nethermind and Besu.
I can find others in the lesser known clients (5% of network) if necessary; but it is unnecessary given that we would encounter near-identical undesirable behavior in smaller EL clients. I haven't looked at the < %5 clients yet - but, again, they will fall to remote exploitation if anyone here deems it necessary.
This report contains attack code for Geth, Nethermind and Besu in their own individual evil clients that pull heavily asymmetric receipts from blocks with 1k+ txs, or through requesting large block header ranges. There are a lot of vulnerabilities in these clients and the Attackathon doesn't give me enough time to find them all (https://i.imgur.com/JarudSi.png) and write massive reports for them - I have found a lot.
Vulnerability Details
It would be easier to TL;DR the exploits. - as to not insult the intelligence of Ethereum programmers and to write fewer massive, organically written reports.
This report could be a single sentence. "Apply rate limiting to the receipts protocol message to thwart botnet attacks". and they'd instantly understand exactly what I mean and how to patch.
The protocol message GetReceiptsMsg isn't rate-limited in any Ethereum client. Naughty low-level P2P attack clients can peer in and DoS/crash from the inside. This is the case with Nethermind, Geth and Besu - and although Geth is the only slow OOM - it jams the EL up and halts P2P comms between the EL/CS layers before even OOM crashing; timing out, e.g. Teku running Geth during a remote attack from 1 to 5 attacking machines depending on machine specs on both ends:
Attack scenario
Bob operates alongside a person or group that oversees sizable botnets, e.g. spam cartel actors, state actors, etc. with xxx,xxx zombies ready to nuke the network by slamming every public listening node with messages that result in network desynchronization, CPU/MEM DoS, crashes and unprecedented netsplits.
He weaponizes a modern P2P botnet that runs the attacks with a modified, unified attack script built specifically to remotely disable the Ethereum network that is able to communicate the state of the attack to itself.
Bob's attack nodes start peering into victim Geth nodes with peer slots available. From there he sends FINDNODE to handshake-ping for UA across every (exponentially) collected peer. Every Nethermind and Besu UA is returned, databased and passed into another routine for quick-crashing them in the future as they crash a lot faster than Geth does. The quick-crashing frees up ~55% of peer slots in Geth nodes, on average, and allow for more attack peers to seed in - slowly - waiting for the command to attack the network simultaneously - at scale, and from the inside while being interpreted as valid heathy peers handling PING/PONG/STATUS/HEADERS/etc. as keep-alive mechanisms.
Much of the network is comprised of Docker instances that reboot clients after they've crashed
At this point - we have a peer list across the network known by distributed evil-client based on their user-agents. These are databased in real-time and are useful for our smartnet's resource distribution.
By running a full attack the only time condition to consider is whether the speed at which the Docker reboots and Geth lag can save the network from a simultaneous DDoS attack sourced by a formidable botnet.
Different machines have different specs, and so will not crash or disconnect from the network at the exact same time - when all threads attacking a node are rejected due to the Ethereum node having been killed, the workload is distributed to immediately focus on, e.g. 32 GB MEM machines versus 16 GB MEM machines that crash twice as fast. The speed at which nodes stress is communicated to the zombies and auto-scaled to redirect or use new resources on peers as they become totally unresponsive, creating network-wide splits, lost funds flying into the void, opens the door to 51% attacks, and so on.
Again, each enode://[hash]@ip:port would have its clients identified by the smart botnet, in addition to the speed at which they become disconnected from the network - something discoverable by evil peers working in concert. The end result is hit or miss finality, funds presumably flying into the void in addition to other undesirable behavior.
The geth crash takes awhile to OOM on a 16 GB MEM machine and the same with even heavier MEM machines. That said - again - before the geth process quits it's already been desynchronized from the network.
Header block range spamming also appears to work against all clients - but all of my scripts can be easily modified to use any other DoS or crash that I've found. I have too many vulnerabilities and not enough time to write reports. I found too many vulns after familiarizing myself with the Ethereum software ecosystem.
Impact Details
51% attacks, total or partial network shutdown, funds lost during failed txs, island networks - netsplits everywhere, operational damage and even after applying a hotfix - reputational damage in permeance.
Attack code (golang, Ubuntu)
Below in Proof of concept, but we'll place it here too:
Technical description of what the attack programs do
With Besu, block header ranges are spammed to trigger an OOM. With Nethermind, receipts from large blocks are requested in a quick way to produce the fastest OOM and/or CPU and/or desync. Geth also requests receipts from these big blocks (1200+ tx) as Nethermind does.
Steps to reproduce the attack
open a .go attack file (below) in a text editor and change the
enode://hash@ip:portto your test victim'senode://hash@ip:portscreen -S attack
ulimit -n 10000
snap install go --classic
nano attack.go
paste from text editor
save as 'attack.go'
go mod init 1go mod tidygo build attack.go./attack
This will not work if all peer slots are in use on a node, which is why we hop other nodes and after UA checking each peer in the network - our botnet attacks accordingly - based on client and later, crashing Nethermind and Besu clients out of Geth's peer list to clear the way for beefier high CPU and port speed machines in the smart botnet.
Outro
I could do this solo by renting a botnet. I know a lot of people think "Ethereum is time and battle tested!" - womp. The problem here is that I haven't, again, had time to find them all - and there are many asymmetric resource-burning networking issues in every client that I've touched.
Given the complexity of this attack - I almost definitely forgot to include at least one item of importance - as there are a lot of moving parts here. But again - a very easy patch - and I'm here to provide clarity on what I've written in addition to what I may have forgotten to write. Apologies for the wordy report - but I think they're required by Immunefi for reasons unknown. I don't think presentation should be so important.
TL;DR Add rate-limiting to block receipt requests. Blocks containing a lot of data hasten the attack.
Thank you all for this opportunity to earn crypto and to help secure the web3 ecosystem. If you run into difficulty running the scripts or reproducing anything I'm here to assist.
AdvancedElephant
🧙🐘
Proof of Concept
PoC: Screenshots and 3 attack scripts for Geth, Besu and Nethermind:
https://www.dropbox.com/scl/fo/vh52nqhqreqrrmvzkk4hd/AC3jGVyfNWOl5dPRdm_g8X0?rlkey=8s0n14a9moq427w1hoolh65jv&st=elis076n&dl=0
Dropbox password: alienbrain
The steps to reproduce running the scripts are in the report. Run them against their associated, fully synchronized EL clients.
Last updated
Was this helpful?