Nodes, Clients, dApps, and centralization risks

On November 11th, parts of the Ethereum ecosystem experienced downtime due to the node provider Infura experiencing a service outage and accidentally hard forking to a minority chain for 30 blocks. This led to crypto exchanges, such as Binance, halting deposits, and many decentralized applications (dApps) built on Ethereum being unresponsive or corrupted. For example, participants in the tBTC protocol for transferring BTC between Bitcoin and Ethereum lost $35k as a result of being stuck on the minority chain.

The outage was caused by a bug introduced by an update to Geth, the node client that Infura uses, in late-2019. This bug was discovered and patched in July 2020, but due to Infura running older versions of the client, the issue had not been fixed on their nodes. The update broke the EIP-211 implementation, which made some gas optimizations related to certain functions in the EVM and was introduced in the Byzantium hard fork in 2017.