Reorg

[LohkoCoder]

Did you check the documentation?

• Yes

Did you check for duplicate questions?

• Yes

Issue Description

Every 12 hours plus 4 minutes, op-node and op-batcher reorg to an unsafe block which was at the height of 12 hours ago, resulting in all transactions losses. Currently op-batcher submits batches every 20 minutes while op-proposer sends proposals every 2 minutes.

Logs

op-node log:
t=2025-07-18T09:09:14+0800 lvl=warn msg="L2 reorg: existing unsafe block does not match derived attributes from L1" err="random field does not match. expected: 0xa1bf65649fea5b7c99724a3751edfee9bd6b7e0e8514cd9e7ff307cbc7667175. got: 0xac71e280016b5b766b6be0b36f3e7fffa284abdb665d6fdfede2eb0c9b01895a" unsafe=0x1bf3fa4a90679269277e08f5026885197ecb717d99c7fd1ef2db7d82176ee375:147474 pending_safe=0x542d7be7d7c5a51091fe17ec0b8afe5e1fe67b8a0110193e84da081d1baa92b2:147473

op-batcher log:
t=2025-07-18T09:09:17+0800 lvl=warn msg="safe chain reorg, clearing channel manager state" syncStatus.headL1=0xfa71b29a0554224ab73e5b47d17de5cbe05cf416fbd4afa111dd444527f52e72:962398 syncStatus.currentL1=0xf10734062bc4548b35d919b66313fd22e455f067db9e6acd3fd58d1ec051739d:962394 syncStatus.localSafeL2=0xd9ab7c07ea47acb9dfc2f6bdf95ff9487a62741fb758bb6db749261fd582ed6b:147479 syncStatus.unsafeL2=0x42ae00f6a45988d3cc4b3604cdebeb4aeece54e03c26c2c354c5f3ecf128d120:147540 syncActions="{clearState:0xc001aef360 blocksToPrune:0 channelsToPrune:0 blocksToLoad:0xc005aa5120}" existingBlock=0xc645cd9adad2dea251b9ce454d7096a7ff42d6adb87a84f7e48ee65b830404ee:147479

op-geth log:
INFO [07-18|09:09:14.219] Imported new potential chain segment number=147,474 hash=944cd4..c7e1e4 blocks=1 txs=1 mgas=0.049 elapsed=1.530ms mgasps=32.260 age=12h53s snapdiffs=188.68KiB triedirty=0.00B
WARN [07-18|09:09:14.427] Large chain reorg detected number=147,473 hash=542d7b..aa92b2 drop=21627 dropfrom=1c48e8..0ee917 add=1 addfrom=944cd4..c7e1e4
INFO [07-18|09:09:15.004] Stopping work on payload id=0x03127e700cbd46ba reason=timeout elapsed=2.000s
WARN [07-18|09:09:16.440] Served debug_traceTransaction conn=128.1.131.139:49380 reqid=0 duration=1.320922674s err="transaction not found"
INFO [07-18|09:09:16.440] Chain head was updated number=147,474 hash=944cd4..c7e1e4 root=92d2dd..b24ee0 elapsed=2.22046504s age=12h55s

Additional Information

op-batcher contract: http://117.50.89.64:3000/address/0x00e95c306d75426ed11729f6899f5bb03510598e?p=1&ps=25
op-proposer contract: http://117.50.89.64:3000/address/0x07fd37b19b7993d56aae6f8e14eda4dda460169a

./op-node --l2=http://localhost:8551 --l2.jwt-secret=../jwt-secret.txt --sequencer.enabled --sequencer.l1-confs=5 --verifier.l1-confs=4 --rollup.config=./rollup.json --rpc.addr=0.0.0.0 --p2p.disable --rpc.enable-admin --p2p.sequencer.key=$SEQUENCER_PRIV_KEY --l1=$L1_RPC --l1.beacon=$L1_BEACON_RPC

./op-batcher --l2-eth-rpc=http://localhost:8545 --rollup-rpc=http://localhost:9545 --poll-interval=1s --sub-safety-margin=6 --num-confirmations=100 --safe-abort-nonce-too-low-count=3 --resubmission-timeout=30s --rpc.addr=0.0.0.0 --rpc.port=8548 --rpc.enable-admin --pprof.enabled --pprof.port=6062 --metrics.enabled --metrics.port=7302 --target-num-frames=8 --batch-type=1 --data-availability-type=blobs --max-channel-duration=1 --wait-node-sync=true --l1-eth-rpc=http://106.75.44.139:8545 --throttle-threshold=500000 --data-availability-type=blobs --log.level=debug --private-key=$BATCHER_PRIV_KEY

./op-proposer --poll-interval=4s --rpc.port=8560 --rollup-rpc=http://localhost:9545 --game-factory-address=0x07fd37b19b7993d56aae6f8e14eda4dda460169a --proposal-interval=10s --l1-eth-rpc=$L1_RPC --num-confirmations=10 --log.level=debug --game-type=1 --allow-non-finalized --private-key=$PROPOSER_PRIV_KEY

./geth --datadir=./datadir --http --http.corsdomain=* --http.vhosts=* --http.addr=0.0.0.0 --http.api=web3,debug,eth,txpool,net,engine,miner --ws --ws.addr=0.0.0.0 --ws.port=8546 --ws.origins=* --ws.api=debug,eth,txpool,net,engine,miner --syncmode=full --gcmode=archive --nodiscover --maxpeers=0 --networkid=707 --authrpc.vhosts=* --authrpc.addr=0.0.0.0 --authrpc.port=8551 --authrpc.jwtsecret=../jwt-secret.txt --rollup.disabletxpoolgossip=true

syncStatus:
cast rpc optimism_syncStatus --rpc-url http://localhost:9545 | jq
{
"current_l1": {
"hash": "0xf9fbc2fbbf510769bef1ecf7353690fd28bf63b99c75ace859e5cca5c129f3ec",
"number": 963114,
"parentHash": "0xad051d5aa92ce939b4200a02bd5c5e5e9e898bc61d3fb781c56b83f61eb620da",
"timestamp": 1752809541
},
"current_l1_finalized": {
"hash": "0x7f1ddeaf1b763553492f6a8daf53bc40c4abd001d8ce17d53da8326ca6d36bc7",
"number": 962994,
"parentHash": "0xb0586c6ff4357a0ebbf123902953fa8b8b4a51f44317a4b67648746bcc26293f",
"timestamp": 1752808101
},
"head_l1": {
"hash": "0xcbfe30cbbd3815bfc0f7628b8663fcc83e319bc1f3ce3a6a1206a81b7b5d68ca",
"number": 963118,
"parentHash": "0x9ccb903b88c564a7d0a1d712d2f97e409246c29f87fbea21a5a9f09ba280ec60",
"timestamp": 1752809589
},
"safe_l1": {
"hash": "0xb9001cc7aa4c31f249634ba1a387bf608549c9bff0de77e28ec3251c15544171",
"number": 963058,
"parentHash": "0xe255b4b0378cae2ab0202743f75d4c3f9593ccc032e6371f8155a8f4e76b2a1c",
"timestamp": 1752808869
},
"finalized_l1": {
"hash": "0x7f1ddeaf1b763553492f6a8daf53bc40c4abd001d8ce17d53da8326ca6d36bc7",
"number": 962994,
"parentHash": "0xb0586c6ff4357a0ebbf123902953fa8b8b4a51f44317a4b67648746bcc26293f",
"timestamp": 1752808101
},
"unsafe_l2": {
"hash": "0x7ae5762931746dfc7d2741fce435fd4b54bcff7a626e093920cac382f84e6d82",
"number": 173427,
"parentHash": "0x5aa41fc5c5bd21565b73731dadd0db05d571d3adbe12f95d338394ccf5bdcf47",
"timestamp": 1752809607,
"l1origin": {
"hash": "0xad051d5aa92ce939b4200a02bd5c5e5e9e898bc61d3fb781c56b83f61eb620da",
"number": 963113
},
"sequenceNumber": 4
},
"safe_l2": {
"hash": "0x037369e6c73e9a3ad1e199990c06e6b0848ae42bb16a2a7fd70b49283362d7d6",
"number": 151799,
"parentHash": "0x5c5f71d3ed7a620ae33b5d2e04ece93eb47042cdc32e9daf3d2ff9898c2bf3e9",
"timestamp": 1752766351,
"l1origin": {
"hash": "0x1f4478ecc9773eb6de9ae871a344e916b17b38755a852b5b432f3707178960dc",
"number": 959514
},
"sequenceNumber": 5
},
"finalized_l2": {
"hash": "0x0829a8839c7605329c007fdc7d1de2ebf26e3727cca342ecae630d02444b3117",
"number": 151079,
"parentHash": "0xf9795a8ef16615ed9f02687a4eeeea4cc99ec5508304da9bf2a7c2ac01281dd0",
"timestamp": 1752764911,
"l1origin": {
"hash": "0x2310bbcaf96fbfd7ea018aa09a805cea16504f0dc56507b1253e338e99f3004f",
"number": 959394
},
"sequenceNumber": 5
},
"pending_safe_l2": {
"hash": "0x037369e6c73e9a3ad1e199990c06e6b0848ae42bb16a2a7fd70b49283362d7d6",
"number": 151799,
"parentHash": "0x5c5f71d3ed7a620ae33b5d2e04ece93eb47042cdc32e9daf3d2ff9898c2bf3e9",
"timestamp": 1752766351,
"l1origin": {
"hash": "0x1f4478ecc9773eb6de9ae871a344e916b17b38755a852b5b432f3707178960dc",
"number": 959514
},
"sequenceNumber": 5
},
"cross_unsafe_l2": {
"hash": "0x7ae5762931746dfc7d2741fce435fd4b54bcff7a626e093920cac382f84e6d82",
"number": 173427,
"parentHash": "0x5aa41fc5c5bd21565b73731dadd0db05d571d3adbe12f95d338394ccf5bdcf47",
"timestamp": 1752809607,
"l1origin": {
"hash": "0xad051d5aa92ce939b4200a02bd5c5e5e9e898bc61d3fb781c56b83f61eb620da",
"number": 963113
},
"sequenceNumber": 4
},
"local_safe_l2": {
"hash": "0x037369e6c73e9a3ad1e199990c06e6b0848ae42bb16a2a7fd70b49283362d7d6",
"number": 151799,
"parentHash": "0x5c5f71d3ed7a620ae33b5d2e04ece93eb47042cdc32e9daf3d2ff9898c2bf3e9",
"timestamp": 1752766351,
"l1origin": {
"hash": "0x1f4478ecc9773eb6de9ae871a344e916b17b38755a852b5b432f3707178960dc",
"number": 959514
},
"sequenceNumber": 5
}
}

Feedback

No response

[LohkoCoder]

Sorry guys, it's my bad. I configured a wrong op-batcher private key, resulting in submitted batch data unrecognized by op-node, which further results in reorg after seq_window_size period.