Cointime

Download App
iOS & Android

AA UserOperation Failure Incident -Why Is Public Mempool Necessary to ERC-4337 Bundlers?

In our previous article, we mentioned several issues that are currently unclear in the Account Abstraction (AA) ecosystem, including a risk in the way Bundler processes UserOperations. The risk is that if a user sends UserOperations with the same nonce to two Bundlers at the same time and the two Bundlers submit the transactions in the same block, one transaction will fail. In this case, the failed Bundler will have paid a small amount of gas cost for nothing.

In this article, I will talk about this issue in detail and it can be inferred that after the public mempool appears, the game state will be changed between Bundlers.

What Happens Exactly

To best showcase the problem, we did a test on Polygon Mumbai following these steps,

  1. Prepare an already deployed contract wallet
  2. Deposit enough fund for gas of multiple UserOperations
  3. Construct a UserOperation with a short calldata and no Paymaster or Aggregator, to keep it simple.
  4. Send this UserOperation with the same nonce to two different Bundlers at the same time. Check the EntryPoint contract history to see the transactions sent by these two Bundlers.

The table below lists some details of the two transactions. (Transaction 1; Transaction 2)

SpecsBundler ABundler B
Transaction Hash0x05d386daa07178fe5607f95fb737436e2366f58895888c713ad45e119e2084f00x74647798233177d8259039b3e5245f1f9f69885db417aa1a7d9ce33728316edb
StatusSuccessFail
From(Bundler)0xb6f005dec7800a7421b2a7a82270d6d7bb7bcd5a0x4337003fcd2f56de3977ccb806383e9161628d0e
UserOperation Nonce33
Sender0xfE3c3675f677A2e61027d949D6A917eCE8057b1b0xfE3c3675f677A2e61027d949D6A917eCE8057b1b
Gas (Matic)0.003501280.0000728
Gas Usage87,53272,819
Repaid from EntryPoint0.003782 
Profit0.00028-0.0000728

                                                                                                          Table 1. Two bundlers send transaction with a same UserOperation

Here is what happened for both bundlers. After receiving the UserOperation, each bundler simulated it based on the current blockchain state. Each was successful because there is enough gas in the wallet contract, calldata is fine, nonce is correct. Based on the gas calculation, this UserOperation is profitable. So they submitted it on-chain in a transaction respectively. 

Next, a problem occurred because the EVM processes transactions in a linear manner, so these two transactions must have a sequence, with one transaction being processed before the other. When the first transaction was completed, the EVM found that the nonce of the UserOperation was incorrect when processing the second transaction, so the second transaction was reverted. Consequently, the two Bundlers obtained completely different results. As shown in the table above, the first Bundler earned 0.00028 Matic, while the second Bundler lost 0.0000728 Matic. The following chart shows the workflow.

                                                                             Figure 1. Flowchart of a same UserOperation processed by two bundlers at the same time

This is just a gas loss from the failed transaction (a bundle) with one UserOperation. What if the transaction includes more UserOperation? Here is another test’s result. Following the same steps of the previous test, except each bundler sent a transaction including two UserOperations instead of one. (Transaction 1; Transaction 2) The first UserOperation of Bundler A is identical from the first one of Bundler B. To be specific, as long as they have the same sender and nonce, the problem happens. We found that the entire transaction failed, which caused the bundler to pay all of the gas for nothing. Compared to the transaction with a single UserOperation, the gas increased by 5408, and the other executable UserOperation in the same transaction was not executed on-chain either. So, this transaction not only caused more economic losses to the bundler, but also caused time losses to another user.

SpecsBundler ABundler B
Transaction Hash0xe321b8c49d3ff8081a0d2589d02c8f71e20f2c0b00f6a6b6575dd2368907b47e0xe310a2c3a15edbed8db7989be4f4066a2fa3ea074c8aae95c42acff14fea0508
StatusSuccessFail
From(Bundler)0xb6f005dec7800a7421b2a7a82270d6d7bb7bcd5a0xfde90001829909d3ae82bec5a5e704d134624c3e
Sender 10xea663B7d161b53Cf7a46035c2790fE19240f5CB20xea663B7d161b53Cf7a46035c2790fE19240f5CB2
Sender 1 UserOperation Nonce1212
Sender 20x4333bF7e6927E38eAf2ED6e72d6956eb659faD640xA71A9a5128C0629FA89d7027FFB7F9eE97D7db1d
Gas (Matic)0.01140.00625
Gas Usage142,67578,227
Repaid from EntryPoint0.0149 
Profit0.0035-0.00625

                                                                            Table 2. Two bundlers send a transaction with two UserOperations, one of which is same

The calldata is the same and pretty simple in the tests here. If those UserOperations were more complex, involved more interacting contracts or had more content themselves, then the gas loss would be even higher. On the other hand, imagine that when AA becomes more popular, Bundlers will bundle more UserOperations together. If a revert occurs, the gas loss will be even greater. The table below shows the gas of the failed transaction containing one duplicated UserOperation of increasing numbers of UserOperations in which the calldata is as same as the tests done above.

No. of UserOps in one transactionGas of the failed transaction
172,819
278,227
383,635 

                                                                             Table 3. Gas of the failed transaction containing one duplicated UserOperation

Dark Forest or Open Field

The vulnerability mentioned above can be exploited as a UserOperation nonce collision attack. And this must be prevented. Here, we have come up with two solutions when there is no public mempool yet.

Pre Public Mempool

A more direct method is for Bundlers to proactively disable the sender or IP address of the UserOperation after a revert occurs. However, there are two issues with this approach. First, banning IP addresses can lead to false positives, and attackers can change their sender and IP address. Second, the user may have unintentionally committed this action.

Another solution that we can think of is to introduce some randomness into the bundling process. Instead of immediately bundling and submitting the received UserOperation to the blockchain, the Bundler could wait for a random number of blocks (e.g., between 0 and 5 blocks) before simulating and bundling it. This makes the attack much less effective, as the attacker cannot guarantee that the Bundlers will bundle the UserOperation in the same block. The cost of the attack increases as the range of the random number increases. This solution may be feasible on high-speed chains, such as Arbitrum, where the block time is relatively short. However, on low-speed chains like Ethereum, where each block takes 12 seconds, the user experience may not be as good. 

The above solutions are temporary before the public mempool is available. While there is a public mempool, the situation will change. 

Post Public Mempool

After the public mempool is introduced, the most UserOperations are from this pool. There must be a mechanism to prevent multiple Bundlers from bundling the same UserOperation in the mempool. Etherspot, who developed the Skandha bundler client, is developing the p2p network of the mempool. According to Etherspot’s information, mempool will be available in August. The UserOperations to be bundled will be transmitted in this p2p network, and once bundled and processed on-chain, they will be marked and delisted. It is meaningless to send duplicate UserOperations to the mempool because it can be easily detected by the p2p network.

When a public mempool appears, multiple Bundlers will have access to the same pool of unconfirmed UserOperations. If the prototype of the mempool is just a version with basic functionalities, there is still a high chance that the UserOperation nonce collision happens when more than one bundler are trying to grab a certain UserOperation into their bundle at the same time. This is harmful for the bundlers community and thus the AA ecology.

As commented from Parthasarathy, the core developer from Etherspot, the p2p interface will be piloted in a chain that supports mev-boost/block builders. This will ensure the elimination of duplicate submissions and reduce online rejections. Marc from Candide team also told us that collaborating with block builders is by design and not optional, one cannot and should not run a bundler in the p2p mempool without a service like mevboost on a public mempool on Ethereum.

If designed this way, the bundler competition in the mempool will not be as disordered as in the dark forest. Combining each bundler's private mempool, bundlers can efficiently process the UserOperations, meanwhile achieve their maximum profit.

We believe that the development of AA will definitely go through a process of discovering and solving problems, and what we are discussing here is just one of them. After going through these developments, AA will eventually be widely adopted. As an innovative web3 infrastructure, we will continue to conduct more research and testing in the field of AA, and hope that more developers will participate in our AA community.

Comments

All Comments

Recommended for you

  • Cook: Apple AI Approved in China, Testing Version Launching Late October

    On October 15, Apple CEO Tim Cook revealed in an interview in Shanghai that Apple AI has been approved in China. A testing version will be launched later in October, featuring initial functions such as real-time translation, writing assistance, and image removal, while Siri AI will not be included.

  • UBS: Fed Expected to Raise Rates for the Third Time in March 2027

    On October 15, UBS stated that it expects the Federal Reserve to implement its third rate hike in March 2027, revising its previous forecast that the rate hike cycle would conclude in December.

  • BTC Surpasses $81,000

    Market data shows that BTC has surpassed $81,000, currently priced at $81,002, with a 24-hour decline of 2.94%. The market is experiencing significant volatility, so please ensure proper risk management.

  • US Spot Bitcoin ETF Sees $261.4 Million Net Outflow Yesterday

    On October 15, according to monitoring data from Farside Investors, the US spot Bitcoin ETF experienced a net outflow of $261.4 million yesterday.

  • Over $476 Million Liquidated in the Last Hour, Primarily Long Positions

    Data shows that in the past hour, a total of $476 million was liquidated across the network, with long positions accounting for $457 million and short positions for $19.7945 million, primarily affecting long positions. Among these, $200 million in ETH and $116 million in BTC were liquidated.

  • ETH Falls Below $2400

    Market data shows that ETH has fallen below $2400, currently priced at $2395.62, with a 24-hour decline of 4.47%. The market is experiencing significant volatility, so please ensure proper risk management.

  • BTC Drops Below $81,000

    Market data shows that BTC has fallen below $81,000, currently priced at $80,989.9, with a 24-hour decline of 2.87%. The market is experiencing significant volatility, so please ensure proper risk management.

  • US Considers Easing Sanctions on Shadow Fleet to Lower Shipping Rates

    On October 15, a US official stated on Wednesday that the Trump administration is considering easing sanctions on vessels known as the 'shadow fleet.' These ships have previously transported oil and petroleum products from countries such as Russia, Iran, and Venezuela. This move aims to reduce the currently historically high global shipping rates. Shortly before this, President Trump had temporarily lifted sanctions on Russian diesel, a decision that drew strong criticism from Ukrainian President Zelensky, several European governments, and some Republican senators. The US official revealed that current discussions focus on whether the US will allow previously sanctioned vessels to transport 'legitimate oil.' Here, 'legitimate oil' refers to crude oil or petroleum products not subject to US sanctions, rather than oil from Russia and Iran that is under sanctions. The official stated that this move aims to facilitate the export of more refined products and crude oil from the Gulf region.

  • Dollar Index Erases All Gains Since CPI Report Release

    On October 15, the Dollar Index (DXY) slowly declined in the Asian session, completely erasing all gains since the release of the U.S. September CPI data yesterday (which had peaked with a cumulative increase of about 70 points), now reported at 102.16. The British Pound against the U.S. Dollar (GBP/USD) and the Euro against the U.S. Dollar (EUR/USD) both rose about 0.3%, currently reported at 1.3231 and 1.1197, respectively. (Jin10)

  • South Korean Stock Index Opens Down Over 5%

    On October 15, the South Korean KOSPI index fell below 6000 points, dropping 5.59% during the day. SK Hynix fell over 7%, while Samsung Electronics dropped over 6%. (Jinshi)