This guide is intended to instruct node operators on the steps needed to successfully transition an EOSIO network through a consensus protocol upgrade (also known as a "hard fork") with minimal disruption to users.
Test networks
Before deploying the upgrade to any non-test networks, protocol upgrades should be deployed and verified on test networks. The version of nodeos supporting the initial set of protocol upgrades is v1.8.1. Existing EOSIO-based test networks can use this version of nodeos to carry out and test the upgrade process.
This test upgrade process can give block producers of their respective EOSIO blockchain networks practice with carrying out the steps necessary to successfully coordinate the activation of the first consensus protocol upgrade feature (or just protocol feature for short), which will fork out any nodes that have not yet updated to the new version of nodeos by the time of activation. The process will also inform block producers of the requirements for nodes to upgrade nodeos to v1.8 from v1.7 and earlier, and it can help them decide an appropriate deadline to be given as notice to the community for when the first protocol feature will be activated.
Testing the upgrade process on test networks will also allow block explorers and other applications interacting with the blockchain to test the transition and the behavior of their applications under the new rules after activation of the individual protocol features. Some of the protocol features (PREACTIVATE_FEATURE and NO_DUPLICATE_DEFERRED_ID as examples) make slight changes to the block and transaction data structures, and therefore force applications that are reliant on the old structure to migrate. One of the protocol features (RESTRICT_ACTION_TO_SELF) restricts an existing authorization bypass (which has been deprecated since the v1.5.1 release of EOSIO) and could potentially break smart contracts that continue to rely on that authorization bypass.
Upgrade process for all EOSIO networks (including test networks)
Because these steps require replay from genesis, after the release of v1.8.1 of nodeos which supports the initial set of consensus protocol upgrades, all node operators should take the following steps as soon as possible. These steps should be followed on an additional node that they can afford to be taken offline for an extended period of time:
- Ensure that their existing node is running the most recent stable release (1.7) of nodeos and then shut down nodeos.
- Make a backup and delete the
blocks/reversibledirectory,state-historydirectory, andstatedirectory within the data directory. - Replace their old version of nodeos with the new release.
- Start the new 1.8 release of nodeos and let it complete replay from genesis and catch up with syncing with the network. The node should receive blocks and LIB should advance. Nodes running v1.8 and v1.7 will continue to coexist in the same network prior to the activation of the first protocol upgrade feature.
A replay from genesis is required when upgrading nodeos from v1.7 to v1.8. Afterward, the v1.8 node can, as usual, start and stop quickly without requiring replays. The state directory generated by a v1.7 node will not be compatible with v1.8 of nodeos. Version 1 portable snapshots (generated by v1.7) will not be compatible with v1.8 which require the version 2 portable snapshots.
Due to the long amount of time it will take to replay from genesis (even longer if running with plugins that track history), block producers of the network are suggested to provide sufficient time to the community to upgrade their nodes prior to activating the first protocol upgrade feature.
Nodes that wish to make the transition but are not interested in tracking the history of the chain from genesis have an option to speed things up by using a version 2 portable snapshots that can be generated by synced v1.8 nodes. Since the portable snapshots are generated in a deterministic and portable manner, users can simply compare the hash of the snapshot files they downloaded from an arbitrary source to the hashes published by a variety of trusted sources, but only if they correspond to snapshots taken at the same block ID.
Special notes to block producers
Block producers will obviously need to run the replay of nodeos on a separate machine that is not producing blocks. This machine will have to be production ready so that they can switch block production over to it when it has finished replaying and syncing. Alternatively, they can take a portable snapshot on the replay machine and move it to yet another machine which is production ready, then activate the switch over from their currently producing v1.7 BP node to the v1.8 node.
Nearly all of the protocol upgrade features introduced in v1.8 first require a special protocol feature (codename PREACTIVATE_FEATURE) to be activated and for an updated version of the system contract that utilizes the functionality introduced by that feature to be deployed. Block producers should be aware that as soon as the PREACTIVATE_FEATURE protocol feature is activated by the BPs, all nodes still on v1.7 will be unable to continue syncing normally and their last irreversible block will stop advancing. For this reason, it is important to coordinate when the activation happens and announce the expected activation date with sufficient time provided to the community to upgrade their nodes in time.
After activation of the PREACTIVATE_FEATURE and deployment of the updated system contract, block producers will be able to more easily coordinate activation of further protocol features. For the remaining protocol features in the v1.8 release, they can activate the features at any time and no preparation time needs to be given to the community since anyone synced up with the blockchain at that time will necessarily be on a version of nodeos that is at least v1.8 and therefore will support the entire initial set of protocol features. Furthermore, due to the PREACTIVATE_FEATURE protocol feature, they can activate the other remaining protocol features with an eosio.msig proposed transaction using the activate action in the new system contract and no replay is required.
The activation of the first protocol feature, PREACTIVATE_FEATURE, however cannot be done with an eosio.msig proposed transaction. It will require more coordination and manual action by the block producers. First, block producers should come to an agreement on the earliest time that they are willing to activate the first protocol feature.
The BPs should then set this chosen time in the configuration JSON file for the PREACTIVATE_FEATURE protocol upgrade of their v1.8 node. Specifically, they should modify the value for the earliest_allowed_activation_time field in the protocol_features/BUILTIN-PREACTIVATE_FEATURE.json file located in the config directory.
It is important that this configuration change happens prior to allowing that node to produce blocks on the network. As long as more than two-thirds of the active block producers have set the same future time in the configuration file for the PREACTIVATE_FEATURE on their BP nodes, the network will be safe from any attempts at premature activation by some other active BP.
After the agreed upon time has passed, any of the active block producers can activate the PREACTIVATE_FEATURE protocol feature with a simple request sent to the producer_api_plugin of their BP node.
To determine the specific format of the request, the digest of the PREACTIVATE_FEATURE protocol feature must first be determined. This can be found by looking at nodeos startup logs, or by sending a request to the get_supported_protocol_features endpoint provided by the producer_api_plugin.
Send a request to the endpoint locally:
curl -X POST http://127.0.0.1:8888/v1/producer/get_supported_protocol_features -d '{}' | jqIn the returned array, find an object that references the PREACTIVATE_FEATURE codename, for example:
...
{
"feature_digest": "0ec7e080177b2c02b278d5088611686b49d739925a92d9bfcacd7fc6b74053bd",
"subjective_restrictions": {
"enabled": true,
"preactivation_required": false,
"earliest_allowed_activation_time": "1970-01-01T00:00:00.000"
},
"description_digest": "64fe7df32e9b86be2b296b3f81dfd527f84e82b98e363bc97e40bc7a83733310",
"dependencies": [],
"protocol_feature_type": "builtin",
"specification": [
{
"name": "builtin_feature_codename",
"value": "PREACTIVATE_FEATURE"
}
]
},
...In this case, the digest of the PREACTIVATE_FEATURE protocol feature is 0ec7e080177b2c02b278d5088611686b49d739925a92d9bfcacd7fc6b74053bd (note that the values may be different depending on the local changes made to the configuration of the protocol features that are specific to the blockchain network).
Then, the local block producing nodeos instance can be requested to activate the PREACTIVATE_FEATURE protocol at its earliest opportunity (i.e. the next time that node produces a block) using the following command:
curl -X POST http://127.0.0.1:8888/v1/producer/schedule_protocol_feature_activations -d '{"protocol_features_to_activate": ["0ec7e080177b2c02b278d5088611686b49d739925a92d9bfcacd7fc6b74053bd"]}' | jqThe above command should only be used after the time has passed the agreed upon earliest_allowed_activation_time for the PREACTIVATE_FEATURE protocol feature.
Any synced v1.8.x nodes can be used to check which protocol features have been activated using the following command:
curl -X POST http://127.0.0.1:8888/v1/chain/get_activated_protocol_features -d '{}' | jqFor example, if the PREACTIVATE_FEATURE protocol feature is activated, that command may return a result such as (specific values, especially the activation_block_num, may vary):
{
"activated_protocol_features": [
{
"feature_digest": "0ec7e080177b2c02b278d5088611686b49d739925a92d9bfcacd7fc6b74053bd",
"activation_ordinal": 0,
"activation_block_num": 348,
"description_digest": "64fe7df32e9b86be2b296b3f81dfd527f84e82b98e363bc97e40bc7a83733310",
"dependencies": [],
"protocol_feature_type": "builtin",
"specification": [
{
"name": "builtin_feature_codename",
"value": "PREACTIVATE_FEATURE"
}
]
}
]
}Once the PREACTIVATE_FEATURE protocol feature has been activated, the new system contract with the activate action can be deployed.
Notes for block explorers, exchanges, and applications
Block explorers, exchanges, and applications building on the blockchain can all follow the four-step processes described above to upgrade their nodes in time and ensure their services continue when the first protocol upgrade is activated. However, they should also be aware that certain protocol features change the behavior of existing operations on the blockchain, and in some cases also slightly change the structure of blocks and transactions.
First, v1.8 changes the structure of transaction traces, even prior to the activation of any protocol features. Clients consuming transaction and action traces made available through history_plugin, mongo_db_plugin, or state_history_plugin should be aware of the changes made to the trace structure (see details at #7044 and #7108). Clients consuming the trace output of the push_transaction RPC from the chain API should not need to do anything since the output of that RPC should be backwards compatible. However, they are encouraged to replace usage of push_transaction with the new RPC send_transaction which uses the new flat structure to store the action traces.
The state_history_plugin has also changed its API and the structure of the files it stores on disk in a backwards incompatible way in v1.8. These changes reflect, among other things, the transaction trace structural changes and the data structure changes made within the chain state database to support the new protocol features. Consumers of the state_history_plugin will need to be updated to work with the new changes in v1.8.
Second, all protocol features are activated by signaling their 256-bit digest through a block. The block producer is able to place the digest of a protocol feature in a special section of the block header (called the block header extensions) that, under the original rules of v1.7, is expected to be empty. This change may especially be relevant to block explorers which need to ensure that their tools will not break because of the extra data included in the block header and ideally will update their block explorers to reflect the new information. The first time block explorers or other consumers of the blockchain data will encounter a non-empty block header extension is during the activation of the PREACTIVATE_FEATURE protocol feature.
Third, upon activation of the NO_DUPLICATE_DEFERRED_ID protocol feature, contract-generated deferred transactions will include a non-empty transaction_extensions field. While block explorers may be interested in exposing the contents of this field in a user-friendly way, clients are free to ignore it. However, for code dealing with the binary serialized form of these transactions directly, they must be capable of successfully deserializing the transaction with the extension data present. Note that this also applies to smart contract code that may be reading the deferred transaction that caused it to execute, whether it is because it is executing an action within the deferred transaction or executing the eosio::onerror notification handler of the contract that sent the (failed) deferred transaction.
Fourth, activation of the RESTRICT_ACTION_TO_SELF protocol feature will remove the authorization bypass that is available when a contract sends an inline action to itself (this authorization bypass was deprecated in the v1.5.1 release of EOSIO). Smart contract developers should ensure their contracts do not rely on this authorization bypass prior to the time the block producers activate the RESTRICT_ACTION_TO_SELF protocol feature, otherwise, their contracts may stop functioning correctly.