PIP-43: Replace tendermint with cometBFT in Heimdall

Authors: Marcello Ardizzone

Type: Core

Table of Contents:

  • Abstract
  • Motivation
  • Rationale
  • Specification
  • Backward Compatibility
  • Security considerations

Abstract

Currently, Heimdall uses a fork of tendermint (called peppermint). The fork, which has been modified to support Polygon PoS specific needs, is based off version v0.32.7.
The refactoring required replaces peppermint with the newer library called cometBFT, which will target version v0.38.x.

Motivation

The primary motivation for this proposed upgrade is to remove tech debt that anchors Heimdall to a Tendermint version that was released around four years ago and is outdated. Despite the continuous upgrades made throughout these years to Peppermint, the upgrade to CometBFT will be beneficial for the PoS network for many technical factors.

Rationale

cometBFT will come with several new features. The main amongst them include:

  1. A new application blockchain interface, called ABCI++, that introduces new methods and types, more flexibility in transactions execution, and a two-phase commit process. Moreover, the blocks execution process is more flexible compared to the linear process of ABCI. Amongst the others, it also introduces advanced functionalities in term of state sync capabilities, allowing for faster and more efficient synchronization.
  2. ABCI++ also comes with a brand new feature, called Vote Extensions. Such functionality will replace the concept of Side Transactions in Heimdall, which was the main driver for forking tendermint. Side transactions are Heimdall transactions whose data comes from external sources (eg: checkpoint txs contain root hash of Bor chain blocks, stateSync txs contain event data from L1 Ethereum). Hence, the data in side transactions needs to be verified by the validators on Heimdall. A VoteExtension is just an arbitrary sequence of bytes that gets added to the block precommit. This means that it is the responsibility of the ABCI++ application (Heimdall in this case) to define and interpret it. In short, vote extensions are basically a device with which validators can vote on data that is extrinsic and agnostic to the consensus protocol.
  3. The upgrade will target a fork of cometBFT which contains the peppermint changes on top of cometBFT v0.38.x. This will allow more frequent upstream merges, which means more functionalities to leverage and a higher level of security.

A comprehensive overview of the new features included within CometBFT can be found under references.

Specification

The proposed solution - which is dependent on other upgrades (such as upgrading cosmos-sdk dependency in Heimdall) - involves several steps to implement:

  1. Port the Polygon PoS changes from peppermint to cometBFT
  2. Upgrade of Heimdall application to a newer version of cosmos-sdk (v0.50.x)
  3. Release of a beta version
  4. Embed the cometBFT into the newer Heimdall
  5. Test the standalone application locally
  6. Audit the code
  7. Possibly rework the code, based on audit results
  8. Release of a stable tag for the Polygon PoS fork of cometBFT

Backward Compatibility

The upgrade depends on a newer version of cosmos-sdk, which makes the code non-backward compatible with the current Heimdall chain. This means the launch of cometBFT within Polygon PoS will only be possible by migrating Heimdall to a brand new chain.

Security Considerations

The upgrade depends on a newer version of cosmos-sdk, which makes the code non-backward compatible with the current Heimdall chain. This means the launch of cometBFT within Polygon PoS will only be possible by migrating Heimdall to a brand new chain.

5 Likes

We are writing here some comments on behalf of the engineering team (part of Informal Systems) that stewards the CometBFT project:

We are fully aligned with the description @marcell033 posted above

A clarification of the relation between Tendermint Core and CometBFT:

  • CometBFT forked from the Tendermint Core repository at the beginning of 2023, and became the canonical Tendermint consensus algorithm implementation for the Interchain (sometimes called Cosmos) ecosystem. The Interchain Foundation (ICF) has been backing CometBFT from day 1
  • The engineering team behind the Tendermint Core repository fully moved to CometBFT at the time of the fork.
  • As of the time of this writing, the Tendermint Core repository is not actively maintained.

Regarding Polygon’s fork of CometBFT for the new version of Heimdall (i.e., the equivalent of peppermint in the current version of Heimdall)

  • Roughly 70% of the contents of the peppermint fork will disappear since – as described by @marcell033 – side transactions can be shipped within vote extensions, which are supported natively by CometBFT v0.38.x
  • The remaining 30% of peppermint needs to stay in the new fork: it mostly contains the implementation of the Ethereum-compatible crypto (uncompressed secp256k1) used by Heimdall.
      - Due to a name collision, this code cannot be upstreamed – as is – to CometBFT at the moment of the Heimdall migration
      - However, there is a strategy in place to upstream this crypto to CometBFT, which will allow Polygon to use “vanilla” CometBFT in a future version of Heimdall after the migration
      - Let us know if you want to know the details of that strategy

There is an additional reason that makes spawning a brand new chain – a hard fork according to CometBFT’s upgrade taxonomy – the only viable alternative. The block format has changed since Tendermint Core v0.32.7, and therefore, the CometBFT databases with legacy data become useless:

  • Some fields in the block have been added/removed
  • The way some hashes are calculated has changed
  • The two bullets above affect things like light-client logic and hash/signature verification
  • TTBOOK, the block data, while available today in the DBs in /var/lib/heimdall/data, is rarely used (especially data from far in the past)
      - Nevertheless, that data can still be available to users by maintaining archive nodes on the current chain (heimdall-137) after Polygon has transitioned to the new chain
  • All this is explained in detail in @marcell033’s PIP-44 initial post

By adopting CometBFT v0.38.x, Heimdall will be benefiting from cool features that were not available as of v0.32.7. Here are some examples:

  • ABCI++, as explained in @marcell033’s post
  • State sync: you can start a brand new node, with an empty /var/lib/heimdall/data directory, and the node, when starting
      - will fetch a recent app snapshot from its peers
      - will install that snapshot into Heimdall app, which will allow you to skip a good deal of blocksyncing (or fastsyncing, as it was previously called)
      - after blocksyncing the last few blocks (those that were produced after the snapshot) the node will start participating in consensus and transaction dissemination right away
      - … as a result, the operator no longer needs to download – or copy – the whole /var/lib/heimdall/data directory (currently ~370 GiB) from elsewhere
  • Advanced pruning. The operator (and the app) has better control on how the /var/lib/heimdall/data directory will grow, via new configuration options.
  • A consensus engine that will be maintained, kept secure, and modernized over time (e.g. we are in the process of releasing v1.0.0)
  • And many more, like proposer-based timestamps (PBTS), the data companion, etc…
2 Likes