dYdX experiences chain halt during scheduled upgrade

The decentralized derivatives exchange protocol dYdX DYDX +3.68% has experienced a chain halt during a scheduled upgrade.

The outage has been ongoing since 6:50 a.m. UTC, according to an incident report. On-chain data shows block production appearing to stop shortly after, with dYdX engineers currently debugging the issue.

The incident represents the first major outage for the protocol since dYdX version 4 and its standalone Cosmos blockchain launched on mainnet in October, according to its status page. 

The dYdX Chain was scheduled to undergo its v4.0.0 protocol upgrade today which could potentially disrupt functionalities. Scheduled maintenance was completed at 6:30 a.m. UTC. However, the chain halt issue began 20 minutes later.

In a status update at 9:21 a.m. UTC, dYdX said the issue continues to be investigated and it has been agreed for network validators to reconvene at 3 p.m. UTC.

“This means that the devs won't suggest a workaround or a fix until then so that the validators won't get jailed for not being online when the chain restarts,” the team said. When a validator is “jailed,” it means it’s been temporarily suspended from participating in the network activities. 

In a further update at 3:32 p.m. UTC, dYdX announced a new software version was available to fix the bug causing the chain halt, with a recommended snapshot for recovery and a note for validator node operators to contact dYdX Operations Services for further instructions or to take part in the co-ordinated effort.

At 4:23 p.m. UTC, the dYdX team said, “A fix has been implemented and blocks are being produced. We are monitoring the results.” The project confirmed the incident had been resolved 30 minutes later.

The incident does not appear to have impacted the price of dYdX's token, which is up over 4% during the past 24 hours, according to The Block's price page.

The dYdX team did not respond to a request for comment from The Block.

Updated with additional details.