MEV On Solana
Hey there!
Today’s piece is authored by the latest addition to our team. Shlok Khemani has joined our team to help with research and writing. Since last December, we have been brainstorming (and learning) from him on all things AI. He brings core expertise in AI and some strong views on Web3 gaming. Readers who enjoy technical writing are in for a treat.
Over the last few years, Shlok has worked with several consumer facing startups in their 0 to 1 journey. His learnings from those experiences will add one more unique lens to how we think of research, markets and scaling ventures. I’m excited to have him with us.
On to the article..
- Joel
March 9, 2024 should have been a great day for Solana validators, stakers, and the Jito Labs team. They had just collected over 10,000 SOL, more than 1.5 million USD, in MEV tips, the highest in a single day to date. Jito’s solution had promised to make Solana fairer and bring the ecosystem more revenue; it appeared to be delivering on that promise. And this was just the beginning of the bull market. Who knew how high these tips could go? Up only.
Instead, March 9 marked the shock announcement that Jito suspended the flagship feature of their product – their mempool. The decision sent ripples through the Solana ecosystem, triggering debates over everything from the wisdom of the move to the fundamentals of the blockchain itself.
After all, it’s not every day that you see a team voluntarily limit a product that was working exactly as intended and making everyone involved bucketloads of money.
In this piece, we cover the context, rationale, and aftermath of Jito’s decision. In doing so, we also talk about MEV on Solana and why this is a pivotal moment for both Jito and the chain. But first, let’s start with a (very quick) primer on MEV.
Forms of MEV
MEV, or maximal extractable value, is the value that ecosystem participants can derive from adding, removing, or reordering transactions within a given block. Not all MEV opportunities are equal; some are considered healthy, while others are debatably less so. Some span a single transaction, while others can persist over multiple blocks. Let’s examine a few common transaction types and the MEV opportunities they present. We’ll keep referring to these examples throughout the piece.
Dex Trade.
A user swaps 1000 USDC for 10 SOL.
Two MEV opportunities arise:
Arbitrage. Suppose the trade occurs on Orca and causes the price of SOL to move from $99.9 to $100. The arbitrageur would want to sell SOL on Orca after buying it from another exchange (where the price remains $99.9).
This type of arbitrage is healthy for markets, as it helps maintain price consistency across different platforms.
Sandwich Attack. If an MEV attacker can access this transaction before it is executed, they sandwich it with two trades of their own– a buy before and a sell after. If executed successfully, this leads to the unsuspecting user getting the worst possible price for their trade while the attacker makes a risk-free profit.
This form of MEV is usually seen as a tax on users (although in some cases healthy for the network).
Let’s sandwich attacks this with an example. Say I want to swap 100 USDC for WIF, currently priced at $0.1. Ideally, I should get 1000 WIF in this swap. But prices on AMMs are dynamic – they can change between the placing and execution of a transaction. To account for this, I also have to set a slippage, which decides the minimum number of tokens I’m willing to accept if there is a price change. If I set the slippage as 1%, I accept getting a minimum of 990 (1000 - 1% of 1000), though I would still prefer 1000 tokens.
Sandwich attackers take advantage of this slippage mechanism. First, they add a transaction to purchase WIF before my trade. This causes the price of WIF to rise just enough for my trade to execute at maximum slippage (I most definitely get 990 tokens). Then, they add another transaction just after my trade where they sell the WIF they purchased initially at the new higher price, pocketing the difference. The profit is risk-free - If they can ensure that these three transactions happen atomically (all or nothing), the profit is risk-free.
Oracle Price Update.
An on-chain Oracle updates the price of SOL to a value that can potentially trigger the liquidation of an overcollateralised loan. Searchers anticipate this price change and race to liquidate the loan, with the first to do so receiving a portion of the collateral as a reward. This MEV is also net-healthy for a blockchain.
A hyped NFT contract is deployed.
Users expect the future floor price of the NFT to be higher than the mint price. Thus, their incentive is to secure as many NFTs as possible before the mint process is over. MEV seekers may attempt to front-run other users by submitting transactions with higher gas fees, ensuring they are processed first. While this may be profitable for the MEV seeker, it can lead to network congestion. Depending on the NFT supply, this opportunity can last over multiple blocks.
The World Before Jito
Solana doesn’t have an in-protocol memory pool (mempool). A mempool is a public waiting area for transactions before they’re written to the blockchain. On other blockchains like Ethereum and Bitcoin, a mempool is typically created by nodes gossiping about transactions they see. Remember that the order in which nodes see transactions can differ due to network latency and location.
So, transactions are not final or ordered until they are included in a block. Searchers (dedicated MEV extractors) can pick up transactions from the mempool and submit bundles (with or without their own transactions) to be executed in a specific order.
On Solana, in contrast, transactions are streamed directly to the current block leader; only they can view the transactions before execution. (Here is a good read on the lifecycle of a Solana transaction)
The first MEV risk comes from the block leader having this asymmetrical information advantage. Because they have access to user trades before execution, they could engage in undesired forms of MEV, such as sandwich attacks. They could also enter into private deals with dedicated searchers for direct access to the transaction stream in exchange for a percentage of profits.
Because block leaders on Solana are chosen in proportion to the stake they hold, such arrangements would lead to richer, more sophisticated validators getting even bigger, creating a centralising effect on the chain.
Now, most validators were not acting in these nefarious ways. We know this because telling signs of such activity, like sandwich attacks, were uncommon on Solana. Yet, the concern remained that as Solana grew in total value locked (TVL) and complexity, the incentives to partake in such deals, if left unchecked, would eventually become too tempting for validators to ignore.
The second form of MEV is very unique to Solana: transaction spamming. Once a block leader executes a transaction that opens up an MEV opportunity – that they themselves haven’t taken advantage of – there is a race among searchers to capitalise on it. The optimal strategy to do this is by spamming transactions. There are two reasons for this.
First, transactions on Solana are very cheap. This allows an MEV extractor to send multiple transactions to capture the same opportunity, and even if most of them fail, the cost is inconsequential compared to the potential MEV profits.
The second reason is related to Solana’s scheduler, which is the part of the client software that decides the order of transaction execution. In some scenarios, the scheduler can execute transactions with lower fees before ones with higher fees. Thus, spamming with low-fee transactions, instead of a single one with high fees, is the best chance to capture the MEV opportunity first.
Such spamming is problematic because it leads to a high number of failed transactions. In most cases, only one transaction can capture an MEV opportunity. However, even after this transaction executes (and the MEV opportunity no longer exists) the validator continues to process other transactions looking to exploit the same opportunity, all of which ultimately fail.
This results in wasted validator resources, reduced network throughput, a suboptimal user experience, and, in the worst-case scenario, a network outage. The Solana Foundation team has implemented multiple protocol-level changes over the years to mitigate the impact of spam, but it remains a thorn in their side.
Enter Jito
Jito is a startup that aims to democratise MEV on Solana. They released the Jito-Solana client, a fork of the default Solana client, introducing new mechanisms to increase transparency, share MEV profits among ecosystem participants, and reduce spam.
Mempool
A block leader’s incoming transactions would be stored in an out-of-protocol mempool and be made public for searchers.Bundles
After scanning the mempool for opportunities, searchers could create bundles of transactions to profit from, attaching a tip (extra fees) to each for prioritised inclusion by the validator.
The Jito client guarantees that the leader executes the bundles sequentially (all transactions in order), atomically (all transactions in the same slot), and all-or-nothing (if a single transaction fails, the entire bundle fails). These guarantees are important for the success of MEV strategies.Block Engine
The block engine filters out invalid bundles and conducts an auction for the remaining ones. The bundles with the highest associated tips are forwarded to the validator for priority execution. Solana produces blocks every 400ms; the auction process occurs in the first 200ms of this period.
You can learn more about the exact mechanics of this MEV solution in our earlier piece on Jito. Let’s revisit the two MEV threats we discussed and examine how these additions aimed to solve them.
Firstly, because a leader’s incoming transactions became public for searchers, their asymmetric information advantage vanished. Now, any searcher can transparently view these transactions and submit bundles to capitalise on the MEV opportunities arising from them.
Second, some transactions that would have resulted in spam had they been executed individually were now parts of bundles that nipped the MEV opportunity in the bud. Take the Oracle price update transaction as an example. The searcher sees the price update in the mempool and immediately bundles it with a loan liquidation transaction. If this bundle is accepted and executed, the MEV opportunity is neutralised within the same block itself, eliminating the incentives for users to engage in spam.
While Jito’s solution helped mitigate spam on Solana, it did not completely solve the issue. Jito bundles only partially fill up blocks; some MEV-inducing transactions still make it through without being included in bundles. Additionally, searchers can only send bundles to validators that choose to run the Jito-Solana client, and not every validator does so. Moreover, for MEV opportunities like hyped NFT mints that span multiple blocks, bundles are not sufficient to prevent spam.
This is because, unlike atomic arbitrages or loan liquidations, these opportunities are not eliminated if captured once; the prize remains available, and rational players will keep spamming till the mint period is over or mint slots are exhausted.
Currently, validators running the Jito client secure over 70% of the total staked SOL. What is their incentive? Financial gain, primarily. The recovery of Solana from its post-FTX slump has seen a resurgence in interest and activity on the chain (triggered, ironically, by Jito’s own airdrop) and, consequently, an increase in the tips searchers are willing to pay. In the week ending March 4, for example, validators earned a total of 42.5k SOL (over $6mn) from MEV tips, in addition to their protocol revenue.
The Suspension
Why, then, did the Jito team decide to stop the mempool party? As it turns out, the mempool may have been serving its purpose a little too effectively, something the network wasn’t prepared for. Let me explain.
Remember how validators could, if they wanted to, sandwich users with the privileged information they had? Most chose not to. But with the introduction of the mempool, searchers were able to access user transactions pre-confirmation with a permissioned API, and they showed no such restraint.
They seized every opportunity they could to make a risk-free buck. One searcher, paid over $300,000 in tips for the inclusion of a single bundle (the strategy ultimately failed).
This situation was exacerbated by the meme coin-induced mania on Solana over the past few weeks. With over 9,000 SPL tokens launching daily, random Twitter accounts supposedly turned a few hundred dollars into life-changing sums of money. Seeing this, everyone wanted in on the action.
Now, because most of these tokens have low liquidity, you have to set a high slippage tolerance to land your trade instantly. Given the FOMO on a token that could be the next Jeo Boden, waiting was not an option, and most traders were willing to accept this trade-off. Some Telegram bots even allowed slippage values to be set as high as 100%!
All of this played perfectly into the hands of searchers, who were sandwiching these high-slippage trades left, right, and center. This is why, on March 8, the day before the mempool was suspended, MEV tips exceeded 10,000 SOL. And it was the users who bore the brunt for this – always getting the worst possible price on their trades.
This was when the Jito team decided to pull the plug on the mempool, citing these “negative externalities.”
The Aftermath
Jito’s bold move is pivotal and has far-reaching implications for validator revenue, network spam, and the future of MEV on Solana.
First, the move risked not only Jito’s own revenue (they keep 5% of all searcher tips) but also that of validators, securing almost three-quarters of the total staked SOL. Most expected MEV tips to drop significantly, and they did for a few days but have since recovered to the pre-suspension levels. This is because Jito suspended only the mempool, not bundles or the block engine; searchers can continue to pay for priority execution for other MEV opportunities like atomic arbitrages.
This recovery in tips also shows that Jito continues to solve a fundamental problem in the Solana ecosystem. The value they’re adding is reflected in Jito’s market capitalization, which has surged in recent weeks. We believe that the market is pricing in the fact that Jito can continue to generate revenue even without the mempool and that if the mempool does ever come back, the revenue will only further increase.
Second, the move coincided with a resurgence of spam on Solana – and it became worse than ever before, with over 70% of all transactions failing on some days. Now, there is a debate in the community over whether the increase in spam directly correlates to Jito’s decision or is a result of a general uptick in activity on the chain.
Our opinion is that it’s a combination of both factors, with the suspension of the mempool playing at least some role. Regardless, if you’ve tried using Solana over the past week, you would have noticed the severely degraded user experience.
Note that there are other solutions in the works to reduce spam, from SIMD-110 (adding exponential fee increases to hot states, similar to EIP-1559), to a new implementation of the scheduler, to changing the network communication protocol the chain uses. Recent changes to the network have made the experience better.
Most importantly, these events have forced the Solana community to confront a looming question: what does MEV on the chain look like in the long term?
One option is to suspend the mempool permanently. However, it is tough to put the genie back into the bottle now that validators have gotten a taste of lucrative MEV profits. Since the suspension, at least three of them have claimed to receive offers from searchers to create private mempools - the very kind of activity Jito sought to prevent in the first place. There was also a public solicitation of an alternate mempool solution on the Jito Discord server in the days following the suspension.
Despite these threats, we haven’t yet seen a reduction in the stake held by validators running the Jito, indicating that validators haven’t widely adopted alternative solutions. This is partly explained by the recovery of searcher tips, meaning validators continue to generate revenue. Another factor is the social pressure and the resulting risk of reduced delegation that a validator would face if they were found to engage in such activities.
The second option is to bring back the mempool. It's worth taking a brief look at Ethereum here. Recall that the mempool is a part of the Ethereum protocol, and unlike Solana, they did not have the option of eliminating it. This means that Ethereum had to adapt to the reality of MEV and has, over time, with the help of teams like Flashbots, created sophisticated solutions like MEV-Boost that mitigate MEV's negative impacts to the greatest extent possible.
MEV-Boost has been widely adopted, regularly proposing close to 90% of the daily blocks produced on Ethereum. These blocks also consistently lead to increased validator revenue compared to locally produced (non-MEV-Boost) blocks, demonstrating their value to the network.
My point is that these problems have been solved before, and though the exact implementations might differ, Solana can draw lessons from Ethereum to guide the creation of a better, more mature mempool mechanism.
This could be an opportunity for Solana to develop a completely new solution to MEV. One idea floated around is for stakers, RPCs, and other validators to socially ostracise validators proven to sandwich users, ignoring their leadership slots. Another is to use a complex cryptographic technique called fully homomorphic encryption (FHE).
The argument for any experimentation is that in the grand scheme of things, Solana has a TVL of around $4 billion, which, while non-trivial, is minuscule compared to the global financial market crypto aims to disrupt. Now is the time to take risks and establish a solid foundation for the long term.
What happens next is anyone’s guess, but it feels like a pivotal moment for both Solana and the broader L1 space. Any blockchain with meaningful financial activity will eventually have to confront the MEV demon. Ethereum has done so in the past. Now, it is Solana’s turn. I’m very excited to see how this unfolds.
To a new beginning,
Shlok Khemani