Concurrency in the eUTxO model is not a problem but a challenge
In the weeks leading up to the start of the Goguen era, with the launch of the Alonzo Hard Fork, there has been criticism of the eUTxO model, designed by IOHK for Cardano.
A “concurrency” issue occurred on the testnet, detected by Minswap developers, who in a blog post, acknowledge that the launch of the public testnet did not meet expectations, arguing that being first is not easy, but it was important to deploy the DEX platform on the public testnet to start collecting data and iterate on designs. After all, that is the purpose of a test network.
For many transactions, the platform would return an inability to process them and you had to try several times until it could “find its place on the block”:
Concurrency
Concurrency is the ability of different parts or units of a program, algorithm, to execute out of order or at the same time, with order, without affecting the final result. In the world of cryptocurrencies, we can better understand concurrency, especially when it comes to smart contracts, as the ability for several different agents to interact with the same contract at the same time.
In the UTxO model, it refers to the number of transactions occurring on demand on a blockchain, and from the same application, by users. In other words, where several transactions “concur” and must all be attended to at the same time, and ordered in their execution.
Cardano’s model returns to the roots of the industry, Bitcoin, but with its creation of the extended UTXO model (eUTXO). Outgoing money leaves the sending address in zero balance and generates a new address to record the unspent balance (if there is a balance greater than zero remaining) of that sending address.
Ethereum’s account-balance based accounting model supports only one address per wallet, with transactions in that wallet, unlike Cardano’s UTxO model, or Bitcoin as well, support multiple addresses and with their multiple transactions per wallet.
In Ethereum’s account-balance model, the state of each block is not stored in the blocks being mined, rather it is stored in nodes locally, which then come to an agreement by comparing ‘StateRoot’ — or the overall state of the system.
The Ethereum Virtual Machine (EVM) interprets transactions as events and determines the state transition outcome of these events based on the previous state. This makes developer abstraction much easier and facilitates the construction of many DApps such as Uniswap, AAVE, Curve.
Smart contracts, themselves are concurrent, but the actual transaction order is still sequential and non-concurrent. What this means is that while Alice, Bob and Eve may be accessing a DEX to exchange pairs of tokens, the miner will still sequentially order their individual transactions as they pack all these transactions into blocks. Then, those blocks are added to the Ethereum blockchain and then the validators re-execute them to ensure that the transactions are indeed correct.
Can Cardano transactions be concurrent?
This now brings us to the current problem that creates a cause for concern among many Plutus developers, the ‘Contract-Concurrency’ problem. This concurrency is different from Ethereum’s concurrency, as currently with Plutus there is no global state or ‘Cardano Virtual Machine’ that all Plutus smart contracts can be connected to. With Ethereum as we mentioned, many different agents can access an Ethereum-based smart contract because of the shared global state, but with Cardano’s UTXO model that is not the case. With Cardano, the contracts themselves are components of the UTXO and essentially act as conditions that govern how the UTXO will be used.
Developers are currently faced with a challenge, as with only one agent that can consume the UTxO, and therefore one contract at a time, which creates the so-called “Concurrency” problem. If we put this in the context of a DEX it becomes more complex because of the amount of currency pair transactions (swap).
The UTxO model is deterministic
Determinism means that a user can predict locally (off-chain) how their transaction will affect the on-chain state of the blockchain. In the context of transaction and script processing, it is a synonym for predictability. This avoids unexpected script validation results or failures, unexpected commissions, and unexpected updates to the blockchain or script state.
In the Shelley, Allegra and Mary eras, the transaction validation process was done in a single step. Alonzo adds Plutus scripts, presenting a two-step validation approach. The result of this strategy is predictable when performing transactions in the general ledger and also ensures fair compensation to nodes for their work and resource usage.
A small amount of uncompensated work is performed by a node to determine that a transaction is not actionable, but without the costly second-phase validation. If the transaction is actionable, the node performs the second phase validation of the transaction. In both cases, the node will be subsequently compensated for both validation phases through the fee or guarantee collected from this transaction. This guarantee is the amount of ADA that will be collected as a fee in case the validation of a script in phase 2 fails.
Critics
Critics argue that the advantages of eUTxO are overstated and the disadvantages understated, calling it IOHK’s central marketing strategy for Cardano.
They argue that the biggest drawback of eUTxO is that it is impossible to automatically compose inputs that share the same state, without the need for a coordinator. Without such a coordinator, this means that in certain use cases, an eUTXO can only be spent once per block.
This means that for a group of users of a given DApp, only 1 of them can perform a transaction every 20 seconds, (how long it takes for each new block to start), because multiple UTxOs cannot be generated on the platform, for the same block. The only solution currently available, they say, is a centralized relay.
The biggest problem, they criticize, is in DEX, as there are a large number of UTxOs for the number of transactions. Decentralized exchange is one of the most important functions in DeFi.
Every time a common state is shared, it needs to be modified (and recorded) in a block, and so an off-chain matcher/batcher system is needed. This is needed with multiple transactions that want to trade the same Liquidity Pool (LP) on an exchange.
It means that either only one transaction per LP every 20 seconds is possible, which is not feasible because of its slowness, or the transactions have to be grouped in the same block with others.
The latter can only be done by using a centralized repeater: transactions are grouped and paired on a centralized server, and then executed as a single transaction on the L1 chain.
They cite as an example, Ergolabs’ centralized server, which compares to the order book and then sends a transaction on their behalf to the Cardano blockchain.
On a decentralized DEX the wallet can post a direct contract call on mainnet without needing anything more than a standard decentralized mainchain client, but they argue that on Cardano, it does not.
On non-eUTxO chains, a DEX transaction is a valid transaction on mainnet, which means it can, quite simply, be published. No servers or backend code are required.
Tight DeFi composability will not work with UTxOs, because shared resources such as AMMs cannot be called on the chain. They require a live backend to intermediate and manage all UTxOs. A big problem for Apps that need to interact within a DeFi ecosystem.
IOHK’s explanations
IOHK explained in a tweet explained that Cardano is an UTxO (Unspent Transaction Output) blockchain, which uses a different programming paradigm for DApps than account-balance based blockchains like Ethereum.
The specific model used by Cardano is the eUTxO (or extended UTxO model) which offers higher security, allows predictability of fees (no nasty surprises…) and offers more powerful parallelization
They clarify that DApps are not limited to 1 transaction per block. If you design the application with multiple UTxOs you can impose more parallelism. This is inherent and is one of the advantages of the eUTxO design as implemented in Plutus.
They told that they will soon release more technical documentation to help developers. The Alonzo HFC event will introduce the core building blocks of Plutus 1.0, at the beginning of the ecosystem’s growth.
Hundreds of developers have completed the Plutus Pioneers course and dozens of projects have already started development on Plutus and are at different stages of experience and preparation. An explosion of activity can be expected in the coming months as the developer community grows in skills and know-how. In the meantime, the availability of tools and frameworks will accelerate, starting with the Plutus Application Backend (PAB). Other community tools and APIs will quickly become available.
Over time, bridges and sidechains and other layer 2 solutions will make developers more comfortable developing for the account-balance model, better known for the blockchain environment.
They ended the tweet by saying that the next few months are going to be the most exciting period in Cardano’s history, with huge activity within the community and the launch of a lot of new projects and that unfortunately, they foresee many more attacks and FUD with concerted attempts to spread misinformation.
IOHK presents a document on How to write a scalable Plutus app.
Finally, they published on the blog an article titled: Concurrency and all that: Cardano smart contracts and the eUTXO model where he explains that Cardano’s approach to DApp implementation is different and therefore requires a different learning curve and approach. This is like working with different programming languages in order to implement a solution, and each language must respect its grammar.
Maximizing concurrency is a skill that must be learned: developers must write code in a way that severely restricts opportunities for contention (e.g., avoiding shared state and accidental dependencies). Then, the system must translate this concurrency into parallelism. Several developers have already identified ways to address this, while others are still developing solutions.
Either way, it is important to understand that to implement a scalable DApp on Cardano, a developer cannot simply use an adapted Ethereum contract. Cardano is based on the UTxO model; it is not account-based. And this means that a single on-chain state will not meet the concurrency property in Cardano. Instead, DApps should split their chain state into many UTxOs. This will increase concurrency in your application, which will enable higher throughput.
The education team has previously shared a simple AMM-style DEX implementation in the Plutus Pioneer course. While this is useful for teaching purposes, this architecture would not directly support a commercial DEX where a sourcebook approach and additional concurrency is required. A developer looking to implement on the Cardano core network would need to enhance the scalability of the architecture accordingly.
Proposals in the Cardano ecosystem
Several development teams individually presented different approaches to address concurrency in the eUTxO model.
Minswap has known the concurrency challenge since they started building in Cardano more than 6 months ago. It is not a fundamental flaw, but simply a design challenge that needs to be addressed.
Minswap has several approaches to address this problem and is weighing the advantages and disadvantages of each solution. The on-chain approaches we have explored are limited by the transaction memory limit and the off-chain ones need a centralized entity or a robust L2 system. Both take time and require a simple and robust DEX first.
Sundaeswap blogged an analysis. Three words you can see coming into the discussion a lot are “concurrency”, “parallelism” and “containment”. Concurrency is the ability of multiple actors to make progress on a task, without interfering with each other. Parallelism is the ability of several actors to make progress on a task at the same time, without interfering with each other. Contention is when several actors actually interfere with each other.
Before talking about solutions, it is worth addressing 3 misconceptions:
Misconception 1: Cardano is flawed because it only allows 1 transaction per block. In fact, the opposite is true. Cardano allows hundreds of transactions per block. Instead, it is correct to say that Cardano allows the output of a given transaction to be spent only once, through a single transaction, so protocols that provide multiple people access to the same UTXO could face contention problems.
Misconception 2: only one user can interact with a smart contract per block/transaction. Also not true; the point of contention is around the UTXO, but many UTXOs can be governed by the same smart contract. This fundamentally comes down to the shift in thinking from Ethereum, where it calls a smart contract to do something , and Cardano, where it blocks outflows with a contract, which determines when they can be spent later.
Misconception 3: The only way to solve this is through centralization. Centralization is one way to solve this problem, but it is not the only way.
For Sundaeswap there seem to be two categories of solutions: design the protocol to tolerate segmentation of its state or add interactions with that state.
One could design a DEX so that it does not require a single liquidity pool. Instead, liquidity is fractured across multiple pools, and the more it is fractured, the more ports there are for people to interact and the less contention over those pools. However, the more the pools are fractured, the less capital efficiency you will have and the greater the value lost in cross-pool arbitrage. The smart part, then, is in designing solutions to those problems: Uniswap v3-style concentrated liquidity, for example.
Alternatively, an order book model for an exchange, which on Ethereum is disastrously expensive to maintain and update, seems more suitable for Cardano, where each order is a separate UTXO. The tricky part, however, is that it still has disputes over orders closer to the current price. A viable solution would be to have the market orders listed in chain, and an external aggregator matches and executes these orders. The smart part, then, is to make sure the aggregator doesn’t have too much power over the market.
Finally, you could create a hybrid exchange, where custody of funds is decentralized and stored on the blockchain, but market making and matching are sent through a central backend server. This solves the engineering problem, but probably makes it a heavily regulated broker-dealer, which brings its own challenges.
The solution for SundaeSwap is different from the above. They tell us they are reserving their design and will reveal how it works very soon.
The team of MELD developers, summarizes in an article some ideas to address the issue.
Splitting UTxOs. If only one UTxO state can be consumed per block, it can be split into smaller UTxOs to spend at the same time. For example, have multiple loan pools for lenders and borrowers to interact per block. However, this solution is usually not capital efficient. A loan pool may not have enough assets for a large borrower to borrow. When lenders want to withdraw their supplied assets, they have to consume all the UTxOs in the loan pool to which they have lent. Both cases require consuming multiple UTxOs to process a request, which requires more UTxOs to avoid congestion, which again reduces the amount of assets in each pool. This does not seem like an easy balance to solve. This design also only works for applications without shared state between the sub-states. It is difficult to adapt applications that do and those that require synchronization or accumulation of sub-states. In certain applications, such as Oracle price consumption, it may be plausible to have multiple UTxOs of similar datum-only.
Batch processing. The MELD architecture belongs to this group of solutions. The main advantages they have identified at this point are, Chained deterministic validation rule, where agents outside the chain can only send the one correct output. This means that there is no influence from any agent outside the chain. Security. No edge exploitation by off-chain agents. Capital efficiency, as you don’t have to pay more for off-chain agents to act with good intentions. Simplicity, where the architecture has good scope. Infrastructure outside the chain is minimal. Complex incentive functions do not have to be designed that can be affected by market conditions and require more governance to adjust correctly.
Deferred updating. The idea is to reserve to create update UTxOs like the batch steps in this architecture. However, instead of constantly applying updates to a state, it waits until that state is required in the chain. For example, in a governance voting process, we only need to consume votes when we consume your proposal to decide whether to pass or fail. Constantly applying votes to the state of the proposal risks congestion and unnecessary transaction fees. The off-chain world is not affected, as it can still find the voting UTxOs and show users the proposal status.
The ErgoDEX developers also brought up how they will treat concurrency and eUTxOs. In an interview on YouTube channel Cardano With Paul they told that they will work on an off-chain code solution with SPOs, particularly with small pools to encourage decentralization. The protocol will run an off-chain bot to choose the validating SPO. An execution fee will be added to the transaction fee on the ErgoDEX platform before the order is subscribed. Then, each validated block with multiple transactions will be moved to the ledger and the pool that runs the winning bot will be rewarded with the execution fee.
The Maladex team wrote a tweet where they explain that in Ethereum everything is absolutely sequential, the pool selects transactions based on reward maximization, then everything is applied to the previous state one by one. It’s very easy to write to something that is absolutely sequential and has global memory, that’s what everyone does when they write their first program, and some developers never deal with concurrency. Cardano really does have concurrency, but it’s up to the developers to design a solid concurrency system with a good user experience.
At Maladex, they say they never thought there was a concurrency problem, but now with all this FUD, they want to shed some light on it. The only thing that is real, yes they are now establishing a lot of design patterns that in a couple of years will be standard for everyone to use.
It’s no different than DeFi taking years to take root on Ethereum, while it took AAVE over two years to show up and then UniSwap.
Mutual Knowledge Systems is proposing an interoperable solution with the Nervos blockchain: AVOUM for Nervos: Account-View-on-UTXO-Model.
The solution will enable secure deployment of “open” contracts on the Nervos blockchain, which is not currently possible, and will bring its smart contract capabilities in parity with Ethereum.
Users can interact with DApps using the convenient account-balance model, popularized by Ethereum, but on a smart contract capable Blockchain using the more robust but more rigid UTxO model, such as Nervos, Cardano, Bitcoin Cash.
The two main problems solved are:
- “open” contracts with an unlimited number of participants or transactions can be subject to economic DoS attacks whereby sophisticated attackers can modify the UTxO of the contract faster than the victims can react, thus blocking their interaction with it. By making these transactions conveniently malleable, intermediaries (on balance, validators) can compete to get transactions accepted by the blockchain in exchange for a fee.
-Writing UTxO unlocking scripts, in which transactions are conveniently malleable, allows users to interact with contracts as if the blockchain uses an account-balance model; but applying the design pattern by hand requires a great deal of discipline and the result may or may not be recognized by transaction posting intermediaries (miners). The solution is to automate this discipline using a suitable Nervos library.
Final words
The design of Cardano was conceived from its origin with two independent layers to better manage the scalability in the processing of its transactions. The Cardano Settlement Layer (CSL) where the ledger lives with its balances and the Cardano Computation Layer (CCL) where all the computation is executed, that is, where all the execution of Smart Contracts takes place.
It was also conceived to execute Smart Contracts, and with a different accounting model from the one Charles Hoskinson himself knew as co-founder of Ethereum.
After more than 5 years of designing Cardano, by the team of IOHK developers, with 116 research papers to date, and academic analysis with peers, when part of the crypto industry criticized for its slow progress to the development of Cardano, wouldn’t it be too rude mistake to have omitted the issue of concurrency, a topic that is basic to the operation of smart contracts.
Beyond the technical issues, which are important, it is always useful to employ common sense, which seems to be the least common of the senses in the crypto industry.
