search
Active Boosts

#43267 [BC-Insight] Potential Indefinite Hang (Denial of Service) in Full Node DA Sync Due to Missing Stream Timeout For Light Node Connection

Submitted on Apr 4th 2025 at 06:54:59 UTC by @Nirix0x for Attackathon | Movement Labsarrow-up-right

  • Report ID: #43267

  • Report Type: Blockchain/DLT

  • Report severity: Insight

  • Target: https://github.com/immunefi-team/attackathon-movement/tree/main/networks/movement/movement-full-node

  • Impacts:

    • Network not being able to confirm new transactions (total network shutdown)

hashtag
Description

hashtag
Brief/Intro

The Movement Full Node's DA communication can hang indefinitely due to missing timeouts when awaiting data on the gRPC stream from the Movement DA Light Node. If the Light Node becomes unresponsive without explicit error (e.g., due to network partition), the Full Node's DA write and read task blocks forever. Importantly, even if the Light Node service recovers and restarts, the Full Node remains stuck because it's awaiting data on the original, now-invalid stream, rendering the node unable to process new blocks.

hashtag
Vulnerability Details

The Movement Full Node interacts with the DA Light Node via gRPC for critical functions: reading DA blocks for settlement and writing transaction batches for sequencing. Both types of operations suffer from a lack of timeouts, leading to potential indefinite hangs.

  1. Reading from DA Stream: The execute_settle::Task (in node/tasks/execute_settle.rs) processes incoming DA data using blocks_from_da.next().await within its main loop:

    // File: movement-full-node/src/node/tasks/execute_settle.rs
select! {
    Some(res) = blocks_from_da.next() => { // <--- Missing Timeout on Read
        // ... process message ...
    }
}

  2. Writing Transaction Batches: The transaction_ingress::Task (in node/tasks/transaction_ingress.rs) spawns background tasks that submit batches and waits for response from the Light Node using da_light_node_client.batch_write(...).await:

    // File: movement-full-node/src/node/tasks/transaction_ingress.rs (within spawned task)
match da_light_node_client.batch_write(batch_write.clone()).await { // <--- Missing Timeout on Write
   Ok(_) => { /* ... success ... */ }
   Err(e) => { /* ... error ... */ }
}

The critical vulnerability is the absence of explicit network-level or application-level timeouts (e.g., tokio::time::timeout) in both the stream read (next().await) and the batch write (batch_write().await) operations. During read, even though there are application level heartbeats from DA node, the full node application doesn't monitor the timely arrival of these DA heartbeats; it simply continues to wait indefinitely on .next().await for any message, rather than acting on the absence of expected heartbeats.

The underlying network connections itself may remain established without generating an immediate error for a long time, depending on the intermediary network devices. As a result, if the Light Node becomes unresponsive or the network connection stalls during either a read or a write operation the respective await call will block indefinitely. Because there are no timeouts, the affected task never exits the blocked state to handle the error or attempt recovery (like reconnection or retrying). This remains true even if the Light Node service subsequently recovers, leading to a permanent hang in either DA read (if stuck on read) or the submission of transactions to the DA layer (if stuck on write response).

hashtag
Impact Details

The chain halts as node completely stops processing new block from the DA layer because the responsible task (execute_settle) is stuck as well is not able to write new blocks to DA.

hashtag
References

Mentioned in details section.

hashtag
Proof of Concept

hashtag
Proof of Concept

The following steps demonstrate that the Full Node's DA sync task gets stuck and does not recover, even when the Light Node becomes available again after a simulated network partition and restart.

  1. Start Environment:just movement-full-node docker-compose local

  2. Simulate Network Partition: Isolate the Light Node from the Full Node

    docker network disconnect movement-full-node_default movement-celestia-da-light-node

At this point the full node is blocked on blocks_from_da.next().awaitas no more logs with Receive DA heartbeat are observed.

  1. Stop Light Node: Stop the Light Node container.docker stop movement-celestia-da-light-node

  2. Restore Network: Reconnect the (stopped) Light Node container to the network.docker network connect movement-full-node_default movement-celestia-da-light-node

  3. Restart Light Node: Start the Light Node container again.docker start movement-celestia-da-light-node

Expected Behavior:

Ideally, after Step 5, the Full Node should either:

  • Detect the old stream is dead (after a reasonable timeout) and attempt to re-establish a new stream to the recovered Light Node, resuming heartbeat/block processing.

  • Error out the execute_settle task, potentially triggering higher-level node recovery logic.

Actual Observed Behavior (based on provided logs):

  • After the Light Node is stopped and restarted (Steps 3-5), the Full Node logs do not show any further Receive DA heartbeat messages or any indication that the execute_settle task has recovered or reconnected. All transactions sent to the node remains stuck as well.

  • The execute_settle task remains indefinitely blocked on the blocks_from_da.next().await call associated with the original, stale stream, effectively causing a permanent halt in DA synchronization for node instance. Similarly batch_writeis blocked.

This demonstrates that the absence of a timeout on the stream's .await call prevents the Full Node from recovering leading to a persistent Denial of Service .

Previous#43255 [BC-Medium] user transactions might be lost due to missing error handling in celestia rpc client requests blob submit failure Next#43287 [BC-Low] Certain fees are unaccounted for causing failed transactions

Was this helpful?