Update: Temporary hotfix deployed to resume the Polygon PoS chain
See the new forum post with the details here.
- Polygon PoS users will likely experience downtime starting at 5:50 PM UTC on March 10th due to an issue with the Heimdall implementation.
- Note that all user funds and data remain fully secure.
- The Polygon team is actively working on a solution and will share an update on ETA as soon as we can.
The Polygon PoS team is currently looking into an issue with the Heimdall implementation which is used by one of the two layers of the Polygon PoS chain. This means the network is currently experiencing downtime which began at 5:50 PM UTC on March 10th.
Here are some further technical details:
- Due to a recent upgrade, the Heimdall node, which is part of the dual-node architecture of Polygon PoS, has halted
- All user funds and data are absolutely safe
- While we’re working on identifying the definitive cause, it seems to have originated from an earlier upgrade consisting of a minor parameterization fix to the Ethereum to Polygon PoS state sync/bridging module
- As part of the earlier upgrade, no consensus/state modules were supposed to be affected, the change what is technically called a
side-handleris for the state sync mechanism
- Although still under investigation by the team, we suspect, there may have been a bug in the upgrade which affected consensus, and caused different Heimdall validators to be on different versions of the chain, thereby not reaching 2/3 consensus. When using Tendermint consensus, this situation will cause the Heimdall chain to halt
- Note that Heimdall does not handle user transactions. It is used for validator related transactions and bridging
- The Bor chain relies on Heimdall for block proposer committee selection, specifically span creation. So once the last span that Heimdall specifies passes, the Bor chain also halts.
- At that time, the Bor chain (or the user-facing Polygon PoS chain) will then only be halted, and while there is a liveness issue, there are no security or safety issues. Meaning that while there will be downtime, state and funds are not affected.
We’re currently working on identifying the definitive causes and preparing mitigation solutions that will resume operation as soon as possible. More details will be shared soon.
Please rest assured these issues are of the highest priority for the Polygon PoS team and we’ll keep you updated as soon as we resolve the issues. You can also follow us on our Twitter handle at 0xPolygonDevs for updates.
Thanks for your support,