0xEmperor

Wildcat: Banking but Better

Thu, 29 May 2025 17:49:54 +0000

Many people believe that Crypto will slowly change the nature of capital markets. While the 2016-2020 period in crypto can be vaguely characterised as an aimless exploration of smart-contract platforms to explore PMF, the DeFi summer of 2020 changed things.

Since the beginning of DeFi summer, we have turbo-run/experimented with various arcs of off-chain finance and learnt several lessons. Most of these experiments were cheap imitations of off-chain finance. We saw the rise and fall of pool-2 farming, Lending protocols, Decentralized exchanges, Perpetual exchanges, Liquidity bonding protocols, Index funds, and more. We also saw several iterations of algorithmic stablecoins, which, in some sense, always broke due to the excess leverage embedded in their system, akin to some of the meltdowns in the financial world. Today, in the decentralized onchain world, only a few of these original protocols survive and have become crucial to the DeFi ecosystem. We can bucket these protocols in several micro granular buckets and abstractions. Still, broadly speaking, the ones that have survived and/or currently thriving are decentralized exchanges, lending protocols, yield trading protocols, perpetual exchanges, and RWA protocols. This is not a comprehensive list by any means, but it broadly covers all grounds.

DeFi has slowly matured, making systems more efficient. While many have repeated these words, a vision of crypto was banking the unbanked, democratizing access to financial instruments and building the future of finance compared to the archaic, slow and rigid system of the current financial world. Talking about the DeFi Renaissance and the second coming of crypto’s original PMF is an undertaking that merits its own article. In this blog, we will be talking about something else.

While we have grown the pie since the early DeFi summer days, there are still many key pieces missing from this story to make it a complete alternative to the financial world.

One core piece that has always been missing from the DeFi landscape is undercollateralized lending. I thought undercollateralized lending would be explored in the coming years in my lists from early 2022 and early 2023 on open problems in DeFi and Crypto. What makes it difficult is, ultimately, an entity is represented as an address onchain. This is all the information you have about its existence. They may have multiple wallets, but none of this adds any helpful information about their credit health. To develop a credit history in the real world, you usually use credit cards often, repay dues on time to get a credit score ascribed to you, and do similar things with borrowing-repayment activities. You are eligible for loans when you have a credit score above a certain level. Better credit scores lead to better loan terms, but they are not fully secured and can be secured by a part of the loan. Undercollateralized loan systems are the heartbeat of the financial world, and when you look at it through that lens, credit is the heartbeat of the modern capital world.

Jan 2022

October 2023

The protocol name from my second list, Wildcat Protocol, is what we will discuss in this blog. I will try to give a brief overview of the protocol itself, its structure, and the various things it allows its users to do. However, around and after this exploration, I would like to weave in the importance of Private credit markets for ecosystems, how they’ve grown and become critical indicators of market health in financial ecosystems, why undercollateralized lending has been challenging, and how Wildcat took a thoughtful yet obvious-in-hindsight approach to bring this instrument on-chain. With Wildcat, we close a loop on DeFi as a decentralized mechanism that utilises blockchains as coordination tools for financial capital among various off-chain parties. In short, it lays the foundation for decentralized finance. Future protocols will be advancements of these building blocks, already seen as order books make their way to on-chain Dexes, better lending markets of different flavours in the form of Morpho and Euler, etc. Future protocols will either be in the spirit of making things more efficient for the base protocol, unlocking more users by making usage seamless with better/advanced UX, or reinventing some of these core cogs in the DeFi machinery with a different lens.

The Allure of Credit: From tradfi to crypto

Before we delve into Wildcat, pondering the nature of credit is worth considering. Credit is more than just financial plumbing; it’s often the invisible hand guiding market sentiment, a powerful indicator of where we stand in the economic cycle. Its expansion fuels booms, inflating asset prices and fostering a collective belief in perpetual motion. Its sudden and sharp contraction signals the hangover, the brutal return to reality. As seen below, something different about this cycle till now has been the absence of credit, which has essentially been credit-starved, which can also be seen in the severe contraction of the Alt market and the expansion of bitcoin dominance in crypto. When credit flows freely, markets soar; when it dries up, they crater, especially when funding smaller parts of the market.

CeFi Lending Market Size by Quarter End - Galaxy Research

This isn’t a new phenomenon. History is filled with examples. The South Sea Bubble, the railway manias of the 19th century, the Roaring Twenties, the dot-com bubble, and the 2008 Global Financial Crisis all bear the fingerprints of credit excess. In the book Devil Takes the Hindmost, the author notes about the 1822-1825 credit cycle: “The boom of 1822-25 can be understood as the product of easy credit conditions; During the boom the restricted growth of credit caused asset prices to rise, stimulating further credit creation”. During credit cycles, “people become convinced the prosperity will last forever”. Declining yields on traditional safe assets and excess capital can make speculative ventures seem particularly attractive.

Different Types of Credit Lines - o3 + Gemini 2.5pro collab

The ascent of private credit from a niche market segment to a significant force in the financial ecosystem has been a defining trend, particularly in the years following the Global Financial Crisis (GFC). Initially, the growth was primarily a response to a shifting regulatory landscape. Post-GFC regulations (like Dodd-Frank and Basel III) imposed stricter capital requirements and risk aversion on traditional banks. This led banks to curtail lending to specific segments, particularly middle-market companies and more complex or esoteric credits, creating a substantial funding gap. This created a vacuum, a “mismatch of supply and demand,” where private credit providers stepped in to fill the financing gap. The persistent low-interest-rate environment following the Global Financial Crisis (GFC) also played a crucial role, compelling institutional investors, such as pension funds and insurers, to seek higher yields, which private credit, often with its floating-rate structures, could offer. Private credit funds stepped in to fill this void, offering an alternative source of capital for businesses that found it increasingly difficult to secure financing through conventional banking channels.

Today, private credit is recognized as a cornerstone of the modern financial ecosystem, not merely as an alternative but as an essential component of capital provision. Its importance stems from its role in financing a significant segment of the economy that might otherwise struggle for adequate funding. For investors, it offers the potential for attractive risk-adjusted returns, portfolio diversification, and current income, often with floating rates that provide a hedge against rising interest rates. Regulatory shifts continue to create opportunities for private credit to expand into new areas as banks step away from certain types of lending, further cementing their role in the financial landscape.

Private credit funds operate with less regulatory oversight and offer bespoke, flexible terms. The market evolved into a multi-trillion-dollar force, driven by institutional investors seeking higher yields in a low-interest-rate environment. However, this growth came with inherent risks, many of which stemmed from the market’s opacity. Valuing private assets is challenging, often relying on less frequent, more subjective models, which can obscure underlying volatility and delay the recognition of losses. This lack of transparency means systemic risks can build up unnoticed until a shock exposes the vulnerabilities.

From ECB’s article “Private markets, public risk?”

Crypto’s recent history provides a hyper-accelerated version of these credit cycles. The CeFi lending boom saw platforms like Celsius, BlockFi, and Genesis become titans, their loan books swelling in lockstep with soaring crypto valuations. At its peak, the combined CeFi and DeFi lending market was valued at over $64 billion. But much of this growth was fueled by practices like massive unsecured loans, rampant asset-liability mismatches, and the acceptance of increasingly esoteric and illiquid collateral. The lack of transparency was staggering; the extent of leverage and interconnectedness across these CeFi giants was largely unknown until they began to topple like dominoes in 2022. The collapse of Terra/Luna, followed by the failures of 3AC, Voyager, Celsius, BlockFi, and Genesis, wiped out tens of billions in value and revealed a system riddled with unsustainable risk. The very entities that marketed themselves as the sophisticated bridge between traditional finance and crypto were often operating with less prudence than a casino. For more, refer to Galaxy Research’s “The state of crypto lending”.

The story of lending onchain is a story of growth. Looking at the following two charts together, it is evident that DeFi lending has ushered into a bigger cycle and is growing, taking over the share/heavy lifting of the role of CeFi lending in a post-FTX world. However, DeFi lending is essentially overcollateralized, so the crypto world still suffers from a credit crunch.

Wildcat: The Protocol

Against this backdrop of opaque CeFi meltdowns and the inherent limitations of purely pseudonymous DeFi, Wildcat emerges.

Today, Credit in crypto is not managed openly. If you have the connections and ability to form deals, you might be able to strike some over-the-counter (OTC) deals in Telegram channels, through private deals, etc. There are no longer credit lines like those that existed in the previous cycle. Wildcat is ultimately an attempt to bring them into the open, on-chain, and let this transparency also support and bolster confidence in investors, sparking a new crypto credit cycle, one that we can see.

Note: I refer to the Wildcat Protocol V2 whenever mentioned in this article.

Wildcat crossed 100M in TVL recently.

As I mentioned, facilitating undercollateralized loans in the onchain world is difficult, exacerbated by anonymity and the general lack of trustworthy information. While explorations into ZK-proofs of identity or credit scores are slowly emerging, there’s another way to approach this: by focusing on institutions. Institutions operate on a different spectrum of trustworthiness compared to the challenge of assessing an individual, anonymous address. They are typically incorporated entities, allowing for basic legal contracts to be written and memorialized onchain, underpinning the legality of any arrangement. While bankruptcy remains a distinct possibility, pursuing recourse for fraud is potentially more straightforward. This institutional focus offers a middle ground between traditional finance’s opaque, audit-reliant systems and the fully transparent but often overcollateralized nature of existing onchain loans. Businesses constantly seek capital to expand, become more efficient, and find alternative credit lines, and there’s a corresponding supply of lenders looking to diversify and achieve specific return profiles.

The failures in the crypto space over the last few years, particularly the collapse of several centralized finance businesses often mired in fraudulent practices, underscore the need for alternatives. While onchain systems are not a panacea, if someone is determined to commit fraud, they will do so; they can offer an additional layer of defence. Transparency can help identify issues before they escalate catastrophically. We’ve often viewed crypto protocols as mere facilitators, but at their core, they are also powerful coordination tools. They can help solve the discoverability problem: where does capital demand meet capital supply? Wildcat steps directly into this, aiming to make on-chain credit more transparent, accessible, and, crucially, customizable.

Observing Wildcat’s mechanics, its approach facilitates direct credit relationships between identifiable, KYB-verified business entities and their lenders, using Ethereum as a transparent settlement and coordination layer. This isn’t about anonymous lending pools; it’s about enabling specific, documented credit lines where the terms are explicit, and all activity is recorded onchain. One might then consider this structure: in practice, it begins to resemble a marketplace for something akin to liquid bonds, with the distinction that the debt instruments are formed through deposits into these onchain vaults. It also becomes a tool to mitigate the daily operational complexities of managing their existing credit lines.

It’s crucial to understand that Wildcat facilitates undercollateralized credit. This means lenders are directly exposed to counterparty risk. While the protocol provides lender protections like penalty APRs for delinquent borrowers and clear mechanisms for managing withdrawals (even if they exceed reserves, using a pro-rata system and a queue), it does not underwrite any loans or shield against defaults. It functions primarily as middleware that coordinates and facilitates these loans. The inherent risk of default often justifies the potential for higher returns.

The power of Wildcat lies in the extensive control it gives borrowers. They can define nearly every aspect of their credit markets: the asset they wish to borrow, capacity, interest and penalty rates, reserve ratios, withdrawal cycle lengths, minimum deposit amounts, whether the debt tokens are transferable, and lockup durations. Borrowers can also manage their markets, adjust parameters (with some protections for lenders, like constraints on APR reduction), and eventually close them by repaying all debt.

This profound customization is made possible by the pre-transaction hook system. Think of hooks as customizable security checkpoints that borrowers can set up at the entrance to their lending markets. More technically, these are tiny, programmable “yes/no” gatekeepers that run checks before any core market action, like a deposit, withdrawal, APR change, or token transfer, hits the main contract. Borrowers select hook templates (covering access control, fixed terms, minimum deposits, or transfer restrictions) from an approved library. When a market is deployed, the “Hook Factory” clones these templates into immutable “hook instances” specifically bound to that market. These instances can enforce various conditions, from a simple OFAC check to requiring a zero-knowledge proof of off-chain revenue, all without bloating the base market contract. Lenders, in turn, can see exactly which hooks guard a market and decide if the baked-in rules suit their risk appetite.

To put it simply, pre-transaction hooks are:

Defined by Borrowers: The creators of the credit market set these rules.
Varied in Complexity: They can be straightforward (e.g., checking a sanctions list) or more intricate (e.g., verifying possession of a specific soulbound token or membership in an approved lender list via a Merkle tree).
Purposeful: They grant borrowers granular control over who can lend to them and under what conditions, effectively gating access or extending the market’s functionality before any funds move or core actions are taken.

For lenders, Wildcat aims to offer choice and transparency. Once a lender obtains the necessary credentials for a market (which can be configured by the borrower to be anything from open access to highly restrictive), the interaction resembles that of a standard DeFi vault. They can assess the terms various borrowers offer—market makers, crypto banking startups, hedge funds, and DAOs. Lenders can simply choose to lend to someone else if the terms aren’t suitable or if a borrower seems too opaque or risky (a critical consideration in a post-FTX world). Wildcat plans to support this with borrower profiles for disclosure and template loan agreements to provide legal recourse in default scenarios. While debt tokens issued by markets are generally tradable on secondary markets, borrowers can also restrict this if needed.

As of 11th May 2025

The range of potential borrowers is broad. Market makers and hedge funds are obvious candidates, as are DAOs and protocol foundations. Today, foundations often raise money through token sales, which can be locked for years or create immediate selling pressure on their asset. As onchain protocols mature and find sustainable revenue streams, an alternative form of credit, where funds are lent at a decided-upon interest rate, will be a game-changer. For instance, if a successful protocol like Hyperliquid decided to raise funds for development through, say, 14% interest vaults on Wildcat, one can imagine significant interest.

It’s possible to view undercollateralized lending as contrary to DeFi’s “don’t trust, verify” ethos. However, as protocols mature, exploring possibilities and pushing constraints is vital. DeFi is not just about transposing the traditional financial world onto a blockchain; it’s also about democratizing access to these powerful instruments. Frankly, what is more decentralized than opening up mass access to one of the most essential components of an economy? Wildcat creates infrastructure by bringing credit relationships on-chain with transparent terms, automatic enforcement mechanisms (within the protocol’s logic), and democratized access. In this system, credit can flow more efficiently through digital currency networks. Whether this represents DeFi’s natural evolution or a hybrid approach blending traditional finance with blockchain benefits is for the market to decide. Ultimately, Wildcat is implemented in a way that allows for extensions, including ZK proofs of assets or reserves on the borrower side, making the protocol more straightforward and, in a sense, future-proofing itself to benefit from technological advancements.

The vision for Wildcat extends beyond serving existing DeFi users. It’s about opening up access to a vital part of the economy for everyone to access. In the future, as tokenization becomes more widespread and more individuals and businesses participate in the decentralized financial economy, it’s conceivable that even a local bakery with access to tokenized euros could crowdfund and manage a credit line on-chain with minimal friction, backed by off-chain legal agreements. It’s about leveraging the transparency and efficiency of blockchain to build a more open and accessible credit system for everyone. Additionally, the ECB notes that opaqueness and strong growth in private markets may give rise to financial stability risks. Valuation of businesses can be a murky art, with assets marked to market less frequently and under more subjective assumptions, potentially concealing losses and actual volatility. Crypto offers a transparent alternative, and as ZK-based financial auditing grows, it will probably become the standard. The only way to participate in private credit.

Against Wildcat: Challenges for Onchain Undercollateralized Credit

The path is not without significant hurdles. Scepticism is warranted.

The core challenge remains: how do lenders accurately assess the creditworthiness of borrowers, even KYB’d ones, in a space still wrestling with information gaps? For all their size, traditional private credit markets face similar issues of opaqueness. Crypto’s past is riddled with platforms that faced difficulties when their borrowers (often SMEs or entities in emerging markets) defaulted. Wildcat’s reliance on borrower disclosure, template legal agreements, and the potential future integration of ZK-proofs for financial data takes the right direction. However, building true, verifiable creditworthiness onchain is an ongoing endeavour.

Another significant challenge lies in the transparency and potential for manipulation within on-chain lending platforms. While on-chain data offers some visibility, it doesn’t always paint a complete picture. For instance, borrowing volumes (and thus perceived platform success or Total Value Locked) can be artificially inflated if borrowers use self-lending loops, repeatedly depositing and borrowing the same capital. While Wildcat aims to provide detailed transaction histories to expose such behaviour, the initial allure of high, albeit manipulated, volumes can be misleading. Furthermore, the on-chain nature of these protocols doesn’t inherently solve the problem of off-chain opacity. The complex, often private, trading strategies and asset/liability structures of large institutional borrowers (like the former 3AC) would largely remain hidden, meaning the protocol might not, on its own, prevent collapses stemming from undisclosed off-chain risks.

I think that’s the ultimate question, would wildcat have stopped 3AC? Unlikely to prevent determined fraud. But it could have made their demise less of a black swan. 3AC’s downfall was exacerbated by the opacity of their borrowings. Had their credit lines been on Wildcat, the terms, repayment schedules, and any defaults or refinancing struggles would have been onchain data points, providing earlier stress signals. Wildcat does not mitigate fraud or avoid it, but given the transparent nature of the protocol and onchain transactions, every transaction would have added more information than was otherwise available. Making debt tokens transferable could also allow for quicker price discovery of distressed assets. The 3AC AUM letter was a prime example of how little real information was available.

Additional challenges include the complexities of cross-jurisdictional legal enforcement—what happens when a borrower in one country defaults on lenders spread across multiple jurisdictions? The reliability of KYB processes also varies significantly, and sophisticated bad actors might find ways to appear legitimate. There’s also the risk of borrower collusion, where multiple seemingly independent entities could coordinate to extract value from lenders.

Conclusion

Credit is a fundamental part of our financial world, from home mortgages to business loans. It’s everywhere. On-chain credit also exists in cryptocurrency, but it often operates behind the scenes—through private deals, exchange-specific services, or opaque OTC arrangements. It’s time to give it some limelight, and Wildcat is an attempt to do justice to it.

The key is building systems that foster transparency, allow for granular risk management, and provide clear avenues for recourse while enabling the essential flow of capital that drives innovation and growth. The shift from public to private markets in TradFi and the parallel universe of crypto’s often-opaque dealings point to the need for better infrastructure.

Wildcat is not a silver bullet. It won’t eliminate risk or single-handedly prevent the next market downturn. It offers sophisticated, transparent, and customizable tools for institutional credit to operate onchain. Allowing borrowers to define their terms and lenders to make informed choices aims to create a more efficient and discoverable market for debt.

In traditional finance, the behaviour of private credit markets, particularly credit spreads and the sheer volume of capital flowing into them, has become a vital barometer of broader market health and investor sentiment. When credit spreads tighten, it signals growing confidence, a willingness to embrace risk, and generally looser lending conditions, often preceding or accompanying economic expansion. Conversely, widening spreads and a pullback in private credit availability indicate rising fear, a flight to quality, and a tightening of financial conditions that can foreshadow economic slowdowns or heightened default risk. The sheer volume of capital flowing into and out of private credit and the terms at which it is offered thus provide valuable insights into the underlying strength of businesses and the prevailing sentiment among institutional investors, often acting as a leading indicator for conditions that will eventually surface in more visible public markets.

This dynamic mirrors the crypto world; the explosive growth of often opaque CeFi lending during crypto bull runs, with its aggressive terms and leverage, directly reflected a period of extreme risk appetite. The subsequent deleveraging and credit contraction during downturns, alongside fluctuating borrowing rates and liquidity on DeFi platforms, similarly act as potent indicators of crypto market health, investor confidence, and the overall availability of speculative capital within the digital asset ecosystem. The challenge and opportunity in crypto credit lies in enhancing the transparency of these flows so they can serve as clearer, more reliable signals for the entire market. And I think Wildcat is the right step in this direction.

The journey of DeFi has been one of rapid experimentation, often mirroring the trial-and-error of financial history, but at an accelerated pace. The primitives that have survived—DEXs, collateralized lending, stablecoins—have proven their utility. Undercollateralized lending, particularly for institutions, has remained the elusive next frontier. With protocols like Wildcat, we are not just building another DeFi application but laying down foundational rails for a more mature, more integrated onchain financial system. The credit must and will flow. Wildcat’s contribution is to help direct that flow with greater clarity, precision, and purpose than before onchain. The rest, as always, will depend on the prudence and diligence of those who choose to participate.

Understanding "An Analysis of Intent-Based Markets"

Wed, 04 Sep 2024 21:08:55 +0000

Notes on the nature of Intent-Based Markets.

In this blog, I discuss the work “An Analysis of Intent-Based Markets” by Tarun Chitra, Kshitij Kulkarni, Mallesh Pai, and Theo Diamandis.

Intents have been touted as a way of expressing generalized transactions on blockchains. Intents allow users to express transactions with a set of conditions or covenants, and only if these conditions are met can the transaction be executed.

Today, the most common sort of transactions on blockchains are “swaps” between two assets. So, let’s consider an example,

“I want the outcome of trading my 1 Ethereum for 3000 USDT.”

This is an example of a simple swap between two assets; you can do it on Uniswap or Ambient today. But you have to make a choice - decide the exchange, the chain, the slippage of your dex, and even check if it’s better off-chain or onchain, considering fees. Intent design abstracts these decisions; the whole process - picking the pool, making accounts/signing transactions, handling transfers (or converting dust in your wallet), etc. will be managed. Additionally, Let’s imagine you instead say, “I want to trade 10 ETH for the best price possible, subtracting fees.” an additional consideration, like considering different venues, now gets added. You must calculate this over all possible routes across chains, making it difficult. While a transaction like this cannot be expressed today, intents eventually aim to provide a framework to accomplish this.

A Solver is an entity that receives your intent and determines how to “solve” it. The solver handles the messy details of trying to optimize for the best possible outcome for you. They try to meet your demands while keeping all the conditions and covenants in mind.

The key is that intents focus on “what you want” rather than “how you want it.” You define the desired results, while someone else determines the “how.” Intents greatly simplify the transaction flow that most users use in crypto by allowing you to specify outcomes and not worry about steps.

For a deeper dive into understanding intents, refer to

An Incomplete Primer on Intents 0xemperor.net

This work is about the nature of solvers, modeling intent-based markets (when they become popular) with different solvers and exploring the dynamics that arise in these solver-driven intent markets.

Decentralized exchanges are essentially passive pools of liquidity between two assets. In an actively traded market, while the global price of the asset moves, DEXes, because of their design, rely on continuous arbitrage to synchronize these prices. In the work Quantifying loss in automated market makers, Millionis et al. consider the market microstructure of AMMs and find the main adverse selection costs that lead to significant losses for the LPs. This is also true in Uni v3, as shown by the authors of Decentralized Finance and Automated Market Making: Predictable Loss and Optimal Liquidity Provision.

Intents are described as a way of credibly executing asynchronous transactions across multiple blockchains. Essentially, they generalize “Request for quote” systems. Request for Quote (RFQ) systems are trading protocols where:

A potential buyer or seller (the requester) asks for price quotes from one or more market makers or dealers.
The requester specifies the asset, quantity, and whether they want to buy or sell.
Market makers respond with their best price quotes.
The requester can then accept one of the quotes or decline all offers.

RFQ systems are often used in over-the-counter markets, bond trading, and some cryptocurrency exchanges. (This explanation clarifies the similarity between intents and RFQs.) Intents are generalized RFQs and allow users to be more granular or specific about the conditions, which are then guaranteed to hold throughout the transaction’s execution.

Intents in Practice

In the world of Intent markets where intents are fulfilled by solvers trying to meet the conditions set by a user. A natural assumption is that solvers will only satisfy a user order if something “in it” for them, i.e., it benefits them. The nature of their setup ( allows them to have information that usual users or unsophisticated market participants do not have. So, incentivizing the solver to execute the user’s orders at optimal prices becomes an open problem and resembles principal-agent problems.

Several intent-based marketplaces exist today, but you might say that the expressivity is slightly constrained given the generality we mentioned before. UniswapX is a popular intent system and has done about 11B in volume at the time of this writing.

UniswapX is a non-custodial, Dutch auction-based trading protocol designed for the Ethereum Virtual Machine. The protocol aims to aggregate both onchain and off-chain liquidity, internalize Miner Extractable Value (MEV) as price improvement, offer gas-free swaps, and support cross-chain trading.

The system’s design revolves around signed offchain orders that are executed and settled onchain. Instead of creating and submitting transactions themselves, swappers sign orders specifying details like input/output tokens, amounts, decay functions, and deadlines. These orders are then picked up by “fillers” (MEV searchers, market makers, or other onchain agents) who submit them to a reactor contract. The protocol uses a Dutch order type, which starts at a price better than the current market price and decays over time. This creates competition among fillers to find the best possible price for swappers while maintaining a small profit margin. The system can be extended to support cross-chain trading, allowing users to trade assets between different blockchain networks. UniswapX also incorporates features like optional filler exclusivity periods, governance-controlled fees, and the ability for interfaces and wallets to charge additional fees. UniswapX is essentially a Dutch auction with a reserve price, which is guaranteed sourced from the univ3 or v2 liquidity pool.

CowSwap is also another intent based protocol that implements the “maximum output” intent and has processed billions since inception. In cowswap, solvers participate in an auction to execute user orders at a uniform clearing price, known as a batch auction.

In the work “Illuminating Ethereum’s Order flow landscape”, the authors show a Sankey diagram that illustrates where solvers source liquidity for transactions that were sent to solver auctions Cowswap, 1inch Fusion, and Uniswap X for the month of November 2023. The figure shows the three solver auctions together have 22 solvers that accessed over 33 liquidity sources and also shows solvers source liquidity across both automated market makers (AMM), most notably Uniswap V3, and private market makers (PMM), most notably, Wintermute and SCP.

Through the diagram, the authors also conclude that millions of dollars in Cowswap user orders were matched with 1-inch user limit orders in landed transactions in November 2023. To win the solver auction, 1-inch limit orders must have provided a better price than private market makers, making a case that expanding market access to fill retail order flow can drive better user outcomes. (This was proposed as a rule by SEC recently, but more on that later).

The authors also talk about “Market maker auctions”, where earlier, the only way to participate in these were permissioned RFQs built by aggregators like 0x and 1inch. Recently, Hashflow developed an RFQ that allowed any project or solver to tap into its market maker liquidity which changed the game. Through Hashflow, market makers filled more volume than other auctions run by aggregators. But why is this important? Market makers that integrated with solvers through Hashflow had two benefits: cowswap required solvers to stake money in USDC and cow tokens, thus introducing a barrier to entry, and cashflow gave a bigger surface area of orders (literally more orders) to fill.

OEV (oracle extractable value) markets also run auctions to sell the right to execute a trade immediately after a price update is sent on-chain via a price oracle. What kind of trades might one want to do with these? For example, if a price update were to make a loan position cross its liquidation threshold on a lending protocol onchain, having access to the guarantee of the trade after might help the entity secure the fees gained.

Going back to UniswapX - one might assume it’s a free lunch at the start and that increased competition might lead to better user fulfillment, but only sophisticated participants can win in the longer term. Does competition always lead to better outcomes for user welfare? How do solver markets change when entry barriers are introduced in the form of required entry fees? How do solver markets change when filling the order requires skill in the form of investment in infra, and thus, some solvers outcompete others on this front? Uniswap had over 2000 addresses participating as fillers in their early days which came down to only 12 addresses by Jan 2024. This also might lead to unintended outcomes since the market might suffer from a lack of competition which may lead to worse outcomes for users.

These are some questions explored and answered in the work “An Analysis of Intent-Based Markets” by Tarun Chitra et al.

An Auction Theoretic Model

Consider a user who wants to swap 1 unit of token $T_1$ for as many units of token $T_2$ as possible. This is referred to as the “maximum output intent.” If you have dollars to convert to ETH or Sol, you want as much quantity as possible (maximum output).

You have two choices:

Trade this on a CFMM like Uniswap or Ambient, which gives you a specific price.

OR
Settle it with one of the $n$ solvers, where each solver $i$ has an inventory that holds the token $T_2$ at a price $p_i$ in token $T_1$.

Naturally, you only want to use a solver if they can give you a better price than the CFMM. To model this well, we have to assume that these prices $p_i$ come independently from a CDF $F(p)$ with density $f(p)$. Explaining these assumptions in short:

Independence: The price that each solver has access to is independent of the prices of other solvers. In other words, knowing one solver’s price doesn’t give you any information about another solver’s price.
Cumulative Distribution Function, $F(p)$: The CDF $F(p)$ describes the probability that a random variable $P$ takes on a value less than or equal to $p$. Mathematically,

$$ F(p) = Pr(P \leq p) $$

Probability Density Function PDF, $f(p)$: The PDF $f(p)$ is the derivative of the CDF $F(p)$ with respect to $p$. It describes the likelihood of the random variable $P$ taking on a specific value $p$. Mathematically,

$$ f(p) = \frac{d}{dp}F(p) $$

Imagine we are drawing random number to represent prices:

CDF, $F(p)$: If $F(100)$ = $0.8$, this means there is an 80% chance that a randomly drawn price will be 100 or less.

PDF, $f(p)$: If $f(50)$ = $0.05$, this means that the likelihood of the price being exactly 50 is low compared to other values.

So now we have a setup for our auction model, where the user runs it to get a better price than the one he could attain from public markets.

Why would solvers decide to participate in an auction? In the auction, The solver with the highest bid fills the order, provided their bid improves upon the public market price $p^*$. The winning solver’s profit is the difference between their true price $p_i$ and their bid. So, the incentive for the solver to win the auction is the spread they can capture on this. Now, Solvers face a variety of choices they need to make:

Do I enter the auction at all?
- Entering an auction might involve entry costs. These represent infrastructure and setup costs.
If I choose to enter, how should I bid?
- After entering an auction, solvers exert effort to find a good price. This effort is costly and may be congestive, i.e., the cost depends on the total number of participating solvers, representing increased competition for scarce liquidity. We can call these congestion/effort costs.

Now, we will look at these cases in detail.

No Entry Costs

A Dutch auction mechanism is where the auctioneer starts with a high asking price and gradually lowers it until a participant accepts it. In the context of the auction, we are running for our order:

User’s Role: The user intends to swap a certain amount of token T1 for as much of token T2 as possible.
Solvers’ Role: Solvers (participants in the auction) bid to offer the best price for this swap.

As we said earlier, Each solver has a price $p_i$ at which they can fulfill the user’s intent. These prices are drawn from a known distribution $F(p)$ with density $f(p)$. The Dutch auction was shown to be equivalent to a sealed bid first-price auction. Now, we try to model the solver strategies as a first-price auction, where the highest bidding solver wins at the highest price.

The highest bid will get filled if it is better than $p^*$. Solvers must decide on a bid $\tilde{p}_i$ that balances the likelihood of winning the auction against the profit margin if they win. The net profit of a solver from being filled is $p_i - \tilde{p}_i$, where $p_i$ is their bid.

A short note on revenue equivalence – Revenue equivalence is a fundamental concept in auction theory. It refers to the principle that under certain conditions, different types of auctions (First-price sealed-bid, Second-price sealed-bid, Dutch or English) will generate the same expected revenue for the seller. The conditions are that the bidders are risk-neutral, they have independent private values, and all bidders have valuations drawn from the same distribution, i.e., they have the same information and bidding strategy. This is a powerful result because it means we can often analyze simpler auction formats and apply the results to more complex ones as long as the conditions of the theorem are met.

If $k$ of the universe of $n$ possible solvers are present in the auction, revenue equivalence yields

$$ \tilde{p}_i = p_i - \frac{\int_{p^*}^{p_i} F^{(k-1)}(x)}{F^{(k-1)}(p_i)} dx $$

The interpretation is that each solver shades their bid without entry or congestion costs. In simple terms, “bid shading” means that a solver will bid less than their true price $p_i$ in the auction. They do this to increase their potential profit margin if they win while still trying to bid high enough to win the auction.

Now that we have a rough idea of what each solver might do. We now model interim profits. Imagine you’re a solver considering whether to participate in an auction to fill a user’s order. You know your price $p_i$, but you don’t know the prices of the other solvers who might compete against you.

Before deciding whether to enter the auction, you’ll want to estimate your expected profit if you participate. This estimated profit is your interim profit. It’s called “interim” because it’s your expected profit after you’ve learned your price but before you know the auction’s outcome.

The interim expected profit of a solver with a price $p_i$ in a setting of $k$ other solvers can be written as:

$$ S(p_i, k) = \int_{p^*}^{p_i} F^{k}(x) dx $$

The Ex-Ante Expected profit, defined as the overall expected profit for a solver considering the distribution of prices and the number of competitors, can be written as:

$$ S(k) = \int_0^\infty S(p, k) f(p) dp $$

where,

$S(p,k)$ is the interim expected profit, and it considers the probability of winning the auction with price $p$ and the resulting profit if they win.
$f(p)$ is the probability density function we saw earlier. Since prices are drawn from a distribution, $f(p)$ represents the likelihood of each price $p$ occurring.

Simplifying we get,

$$ S(k) = \int_{p^*}^{\overline{p}} F^k(p) (1 - F(p)) $$

where

$p^*$ represents the lower bound of the integral, possibly the public market price.
$\overline{p}$ represents the upper bound of the prices considered in the market

What did the simplification yield? We transformed the original integral of the expected profit into a form that explicitly accounts for the probability of a price being higher than the public market price $p^*$. It reveals how the distribution of the prices and the cumulative effect of multiple bidders impact the expected profit of the solver.

Why are we calculating interim profits $S(k)$?

Entry decisions: A potential solver will compare their expected interim profit $S(p_i, k)$ to their entry cost $c_i$ to decide whether to enter the auction. If $S(p_i, k) > c_i$, they expect to make a profit by entering, so they will participate. This is the key condition that determines the equilibrium number of entrants $k^*$.
Bid shading: The interim profit function $S(p_i, k)$ is used to derive the optimal bid shading strategy in the Dutch auction. Solvers shade their bids to balance the trade-off between a higher profit margin if they win (incentive to shade more) and a higher probability of winning (incentive to shade less).
The user’s expected welfare from the auction is directly related to the solvers’ interim profits. In particular, the user’s expected payment is equal to the second-highest solver’s expected interim profit. So understanding how interim profits vary with the number of solvers $k$ and the price distribution $F(p)$ is crucial for assessing the user’s welfare.
The paper examines the impact of entry costs and congestion costs on interim profits. This analysis helps in understanding how these costs influence the solvers’ decisions and the overall market dynamics.

The interim profit function is a key building block of the model that connects the solvers’ entry and bidding decisions to the overall competitiveness and efficiency of the intent-based markets.

The paper considers three types of distribution from which the prices can be derived: Exponential, Uniform, and Pareto. The above equation $S(k)$ is simplified with the help of the PDFs of these probability distributions, and we get a final equation in terms of $k$. These equations help us model-specific cases like entry and high effort costs and gain insight into how many solvers might eventually be left or how they slowly change because of various costs. These costs influence the equilibrium number of solvers ($k^*$) participating in the auction.

Exponential Distribution:
- The exponential distribution is often used to model the time between events in a Poisson process.
- The probability density function (PDF) is $f(p) = \lambda e^{-\lambda p}$, where the $\lambda$ is the rate parameter.
- An exponential distribution means many solvers have relatively low prices, but occasionally, a solver might offer a much higher price.

After simplifying with PDF, we get,

$$ S(k) = \frac{1}{(k + 1) \lambda} $$

Uniform Distribution:
- The uniform distribution assumes all outcomes are equally likely within a specified range.
- If prices are uniformly distributed between 0 and 1, the PDF is $f(p)=1$ for $0 ≤ p ≤ 1$.
- A uniform distribution means prices are evenly spread out within some range.

After simplifying, we get,

$$ S(k) = \frac{1}{(k + 1)(k + 2)} $$

Pareto Distribution
- The Pareto distribution is often used to model distributions with heavy tails, such as income distributions.
- The PDF is $f(p) = \frac{\alpha x_m^\alpha}{p^{\alpha + 1}}$ for $p \ge x$.
- If the prices solvers offer follow a Pareto distribution, a few will offer very high prices, but most will offer lower prices.
- After simplifying, we get $S(0) = \infty$. This means that with no other solvers, the expected interim profit is infinite, which is a characteristic of distributions with heavy tails.

Costly Entry

In this scenario, we’ll assume that every solver must pay an entry cost $c_i$ if they bid. What might this represent? It could be real-world expenses like setting up infrastructure or the cost of developing/licensing the necessary software to participate in Intent markets. The costs are distributed according to a distribution $F_C$.

Entry costs help us understand participation by providing a threshold cost $\bar{c}$. If the costs exceed the threshold, the entrant is not profitable. So naturally, the maximum cost solver is the one who exactly meets the threshold. We can write this as

$$ \sum_{k=0}^n (\binom{n}{k} F_C^k(\bar{c})(1-F_C(\bar{c}))^{n-k})S(k) = \bar{c}. $$

Once you have a threshold $\bar{c}$, the expected number of entrants into the solver market can be computed as:

$$ K_* = n F_C(\bar{c}) $$

where $F_C(\bar{c})$ is the cumulative probability.

In the intent market model context, we have $n$ potential solvers, each independently deciding whether to enter the market. The probability of a solver entering is $F_C(\bar{c})$, which is the cumulative probability that their cost is below the threshold $\bar{c}$. Since each solver’s decision is independent and has the same probability $F_C(\bar{c})$ of entering, we can apply the binomial distribution property directly. This property states that the expected number of successes (entrants) in $n$ independent trials, each with success probability $p$, is $n \times p$. Therefore, the expected number of entrants $k^*$ is calculated as $n \times F_C(\bar{c})$, where $n$ is the total number of potential solvers and $F_C(\bar{c})$ is the probability of each solver entering. This calculation assumes that each solver’s decision to enter is independent of the others, a fundamental property of the binomial distribution.

Exponential Distribution

For the case of exponentially distributed prices with rate $\lambda$, the expected number of entrants $k^*$ is derived as follows. The left-hand side of the equation determining the threshold cost ($\bar{c}$) is:

$$ \sum_{k=0}^{n} \left( \binom{n}{k} F_C^{k}(\bar{c})(1 - F_C(\bar{c}))^{n-k} \right) S(k) = \frac{1}{(n+1)\lambda F_C(\bar{c})} (1 - (1 - F_C(\bar{c}))^{n+1}) $$

This leads to the equation:

$$ \frac{1}{(n+1)\lambda} = \frac{\bar{c} F_C(\bar{c})}{(1 - (1 - F_C(\bar{c}))^{n+1})} $$

Using Taylor approximations for large $n$, it is derived that $\bar{c} = O(1/\sqrt{n})$, implying $k^* = O(\sqrt{n})$. The analysis shows that the number of entrants grows more slowly than the total number of potential solvers, indicating that entry costs significantly reduce the number of solvers willing to enter the auction. Specifically, $k^* = O(\sqrt{n})$, meaning the number of entrants grows roughly with the square root of the total potential solvers.

Uniform Distribution

For prices uniformly distributed between $[0, 1]$, the left-hand side of the equation determining the threshold cost $\bar{c}$ is:

$$ \sum_{k=0}^{n} \left( \binom{n}{k} F_C^{k}(\bar{c})(1 - F_C(\bar{c}))^{n-k} \right) S(k) = \\ \frac{1}{(n+1)(n+2)F_C^2(\bar{c})} (1 - (1 - F_C(\bar{c}))^{n+2} - (n+2)F_C(\bar{c})(1 - F_C(\bar{c}))^{n+1}) $$

This leads to the equation:

$$ \frac{1}{(n+1)(n+2)} = \frac{\bar{c} F_C^2(\bar{c})}{(1 - (1 - F_C(\bar{c}))^{n+2} - (n+2)F_C(\bar{c})(1 - F_C(\bar{c}))^{n+1})} $$

Similar arguments show that $\bar{c} = O(n^{-2/3})$, implying $k^* = O(n^{1/3})$. This indicates that the expected number of entrants $k^*$ grows even slower than in the exponential case, further highlighting the impact of entry costs on reducing the number of solvers.

Pareto Distribution

For the Pareto distribution with infinite mean, the expected profit $S(0)$ is infinite, leading to:

$$ (1 - F_C(\bar{c}))^n S(0) \leq \bar{c} $$

Since $S(0) = \infty$, the only solution is $\bar{c} = \infty$. Therefore, $F_C(\bar{c}) \to 1$ as $\bar{c} \to \infty$, implying $k^* = n$. This means that all potential solvers may enter the market due to the anticipation of large prices. This case is unique because the expected profit for a single solver with no competition is infinite.

Extreme spacings refer to the gaps between the highest and second-highest values in a sample. In the context of the Pareto distribution, these spacings can be quite large due to the distribution’s heavy tail. In auctions, extreme spacings can lead to significant differences between the winning bid and the second-highest bid. This can result in substantial bid shading, as solvers may strategically bid lower to increase their expected profits.

The Pareto case, while allowing full participation, may still not provide optimal outcomes for users due to potential bid shading behavior by solvers. This is due to the user’s realized price being far below the highest potential price. Significant bid shading may limit the user’s ability to realize large welfare increases, as the ratio of the expected second-highest price to the expected highest price approaches zero as $n$ increases:

$$ \frac{E[p_{n-1:n}]}{E[p_{n:n}]} \to 0 \text{ as } n \to \infty $$

This suggests that even with a large number of solvers present in the auction, the user can realize a price that is only a small fraction of the theoretical highest price available to any of the solvers.

Costly Effort

In the previous scenario, we considered the impact of entry costs on market participation. Now, let’s delve deeper into the dynamics within the market itself. Once solvers have entered the auction, they face another challenge: the need to invest costly effort to find the best prices to win and satisfy the user’s order. This effort modifies the distribution of prices to $F(p,e)$, where $e$ represents the effort level chosen by the solver. The effort is not only costly but also congestive, meaning it increases with the number of competitors. This is modeled by $c(k^*, e)$, where $c$ is a cost function that is increasing and convex in both the number of entrants $k^*$ and the effort level $e$.

Ultimately, what we explore here is the consideration of how solver effort affects their ability to offer competitive prices and how this effort might be influenced by the number of participants in the market (congestion effects).

(Note: people who don’t want to read the math can go to the conclusions in bold.)

Effort-Dependent Prices and Costs

Suppose $k^*$ solvers have entered the auction. Each solver $i$ chooses an investment level $e_i \in \mathbb{R}^+$, influencing the price $p_i \sim F(p, e_i)$. Higher investments lead to better prices, satisfying $F(p, e) \leq F(p, e')$ for $e > e'$.

Cost of Effort and Congestion

The cost of effort $e$ with $k^*$ entrants is $c(k^*, e)$, increasing and convex in both arguments. A special case is no congestion cost when $c(\cdot)$ is constant in $k^*$.

Equilibrium Effort

To find the equilibrium investment $e^*$, consider solver $i$, assuming other $k^* - 1$ solvers draw $p \sim F(x, e^*)$. The expected revenue for solver $i$ with price $p$ is:

$$ S(p, k^*, e^*) := \int_{0}^{p} F(x, e^*)^{k^*-1} dx $$

Given this, solver $i$ must choose their investment level $e_i$ to maximize their expected profit:

$$ e_i = \arg \max_{e \in \mathbb{R}^+} \int S(p, k^*, e^*) f(p, e) dp - c(k^*, e) $$

The first-order condition at $e = e^*$ is:

$$ \left( \int S(p, k^*, e^*) \frac{\partial f(p, e)}{\partial e} dp - \frac{\partial c(k^*, e)}{\partial k} \right)_{e=e^*} = 0 $$

This defines $e^*$ as a function of $k^*$, allowing us to analyze how congestion and effort impact net revenue.

Specific Example: Uniform Distribution

Consider a specific example where $F(p, e) = p^e$ for $p \in [0, 1]$. This implies $f(p, e) = e p^{e-1}$. Suppose the cost function is $c(k^*, e) = \alpha(k^*) \frac{e^2}{2}$, where $\alpha(\cdot)$ is an increasing function that parametrizes the ‘congestiveness’ of the market. The expected revenue becomes:

$$ S(p, k^*, e^*) = \frac{p^{(k^*-1)e^*+1}}{(k^*-1)e^*+1} $$

Substituting this into the first-order condition, we get:

$$ \int_{0}^{1} \frac{p^{(k^*-1)e^*+1}}{(k^*-1)e^*+1} (p^{e^*-1} + e^* p^{e^*-1} \ln p) dp - \alpha(k^*) e^* = 0 $$

Solving this integral yields:

$$ \frac{1}{(1 + e^* k^*)^2} - \alpha(k^*) e^* = 0 \implies \alpha(k^*) e^* (1 + e^* k^*)^2 = 1 $$

Since $\alpha(\cdot)$ is nondecreasing, $e^*$ must be decreasing in $k^*$. If $\alpha(k^*)$ is linear $\alpha(k^*) = \Theta(k^*)$, $e^* k^*$ is constant in $k^*$. If $\alpha(k^*)$ grows faster (slower) than linear, $e^* k^*$ is decreasing (increasing) in $k^*$. For $k^* = \omega(1)$, this implies $e^* = o(1)$ in $k^*$, indicating vanishing expected equilibrium effort as $e^* k^* \to 0$.

This means that as the market becomes more congested, solvers will invest less effort in finding better prices, leading to a lower equilibrium effort level.

Welfare as a Function of Effort

User welfare is maximized when the expected revenue of the auction is maximized, as it is the (expected) price the user receives. By revenue equivalence, the expected revenue equals the revenue of a second-price auction:

$$ \text{Rev}(e^*, k^*) = \int_{0}^{1} p k^* (k^*-1) (1 - F(p, e^*)) f(p, e^*) (F(p, e^*))^{k^*-2} dp $$

For the specific example, this becomes:

$$ \text{Rev}(e^*, k^*) = \frac{e^* (k^*-1)}{1 + e^* (k^*-1)} \times \frac{e^* k^*}{1 + e^* k^*} $$

Two claims are made about this formula:

If $\alpha(k^*) = o(k^*)$, then $\text{Rev}(e^*, k^*)$ increases in $k^*$. This means that if the congestion cost is sublinear in $k^*$, allowing more solvers to enter the market benefits user welfare. Each additional solver contributes to a higher total revenue without significantly increasing the cost per solver.
If $\alpha(k^*) = \omega(k^*)$, then $\text{Rev}(e^*, k^*)$ decreases in $k^*$. This means that if the congestion cost is superlinear in $k^*$, allowing more solvers to enter the market is detrimental to user welfare because the increased cost per solver outweighs the additional revenue generated, leading to a lower total revenue.

Essentially summing up:

In markets with low congestion costs, encouraging more participation (higher $k^*$) can improve user welfare.
In markets with high congestion costs, restricting entry might actually improve user welfare by incentivizing higher effort from each solver.

This analysis shows that the presence of effort costs can lead to reduced user welfare due to congestion, and a welfare-maximizing planner may prefer to limit entry into the market. The specific example with a uniform distribution illustrates how the equilibrium effort $e^*$ and user welfare $\text{Rev}(e^*, k^*)$ is influenced by the number of entrants $k^*$ and the congestion cost function $\alpha(k^*)$.

Since this model is not as intuitive as the next one, here is an analogy about chefs cooking in a competition

Imagine a cooking competition where chefs (solvers) compete to sell a meal to a single judge (user). Each chef has a unique set of ingredients and recipes that determine the quality of the meal they can prepare. The distribution of meal qualities across chefs is represented by $F(p)$, similar to the distribution of prices across solvers in the intent-based market.

Before the competition begins, each chef must decide whether to pay an entry fee (entry cost) to participate. These entry fees, denoted $c_i$, vary for each chef and are drawn from a distribution $F_C$. This is analogous to the heterogeneous entry costs faced by solvers, such as infrastructure or setup costs. The entry fee is a sunk cost once paid, influencing the chefs’ decision to participate based on their expected return from the competition.

After paying the entry fee and observing their own meal quality $p_i$, each chef decides how to price their meal in the competition. They consider their expected profit (interim profit $S(p_i,k)$), which depends on their meal quality and the number of other chefs they expect to compete against $k$. This is like the solver’s decision-making process.

In equilibrium, chefs will enter the competition until the expected profit for the marginal chef equals the entry fee. This determines the equilibrium number of chefs $k^*$ who participate,

Once the competition begins, chefs can exert effort to improve their meals, but this effort is costly. Moreover, if many chefs are in the competition, they may face congestion in the kitchen, reducing everyone’s efficiency. This is similar to solvers incurring effort costs to find better prices and facing congestion when many solvers are searching simultaneously.

During the competition, chefs may engage in “bid shading” by pricing their meals below their true quality. They do this to increase their chances of winning while still making a profit. This is analogous to solvers shading their bids in the auction to balance the trade-off between winning probability and profit margin.

The judge’s welfare, like the user’s welfare in an intent-based market, depends on the quality of the winning meal and the price they pay for it. This welfare is determined by the chefs’ entry and pricing decisions, which are shaped by the same economic forces as in the solver market: entry costs, effort costs, congestion, and strategic bid shading.

The key insights from the auction model hold in the cooking competition analogy:

Entry costs can lead to limited participation and reduced competition among chefs.
Costly effort and congestion can result in chefs underinvesting in meal quality, similar to solvers underinvesting in price improvement.
Chefs will strategically shade their prices based on the level of competition they face, just like solvers shading their bids.

Optimization Model

The probabilistic model presented in the previous sections provides a robust framework for understanding the strategic behavior of solvers in intent-based markets. However, it relies heavily on assumptions about the distribution of prices and the costs associated with entry and effort. While this approach is powerful, it may not capture the full complexity of real-world scenarios where solvers exhibit heterogeneous behaviors and utilities. An alternative deterministic approach, grounded in optimization theory, offers a complementary perspective. This approach allows for explicitly modeling utility and cost functions, which can be tailored to specific data and scenarios, providing a more flexible and potentially more accurate representation of solver behavior.

Framework and Assumptions

The deterministic model begins by assuming that each solver $k$ is equipped with a utility function $u_k$ and a cost function $c_k$. The utility function reflects the solver’s preference for holding tokens, while the cost function captures the expenses incurred in providing liquidity or executing trades. The user’s utility is derived from the maximum output of token $T_2$ achievable using a combination of solvers and on-chain Constant Function Market Makers (CFMMs) encapsulated in an aggregate forward exchange function $G$.

Essentially, Each solver $k$ , is characterized by the following:

For a given amount $x$ of token $T_1$, the solver provides $\alpha_k x$ of token $T_2$.
The solver incurs a cost $c_k(x)$ and gains utility $u_k(x)$.
Cost function $c_k: \mathbb{R}_+ \to \mathbb{R}_+ \cup \{\infty\}$ is convex.
Utility function $u_k: \mathbb{R}_+ \to \mathbb{R}_+$ is concave.
Both functions are defined over non-negative reals and are closed and proper.

The net utility for solver $k$ is given by:

$$ U_k(x_k) = -\alpha_k x_k - c_k(x_k) + u_k(x_k) $$

Where this utility is measured in terms of token $T_2$, which serves as the numeraire.

In the user model, The user wishes to trade $\delta$ units of token $T_1$ for a maximum amount of token $T_2$, using a combination of solvers and on-chain Constant Function Market Makers (CFMMs). The model introduces an aggregate forward exchange function $G: \mathbb{R}_+ \to \mathbb{R}_+$, which maps an input of token $T_1$ to the maximum output of token $T_2$ using all on-chain CFMMs. This function is concave and nondecreasing.

The user’s utility is given by:

$$ U_p(x) = G(\delta - \sum_{k=1}^K x_k) + \sum_{k=1}^K \alpha_k x_k $$

This framework’s central problem is maximizing social welfare, which is defined as the sum of the user’s utility and the solvers’ net utilities. This problem can be formulated as:

$$ \text{maximize} \quad G(\delta - y) + \sum_{k=1}^{n} u_k(x_k) - c_k(x_k) \\ \text{subject to} \quad y = \sum_{k=1}^{n} x_k $$

Here, $y$ represents the total amount of token $T_1$ traded, and $x_k$ is the amount the solver $k$ traded. The objective function seeks to maximize the user’s utility from trading $\delta$ units of the token $T_1$ while accounting for the solvers’ utilities and costs.

Dual Approach and Dutch Auction Mechanism

The Lagrangian of the social welfare maximization problem is:

$$ L(x, \tilde{x}, \nu) = G(\delta - y) + \nu y + \sum_{k=1}^{n} u_k(x_k) - c_k(x_k) - \nu x_k $$

The dual function $h(\nu)$ is then:

$$ h(\nu) = \sup_{y \geq 0} \{G(\delta - y) + \nu y\} + \sum_{k=1}^{n} \sup_{x_k \geq 0} \{u_k(x_k) - c_k(x_k) - \nu x_k\} $$

The dual problem is to minimize $h(\nu)$, which can be solved iteratively using a method akin to a Dutch auction. Starting with a high price $\nu$, the mechanism queries each solver for their optimal trade $\tilde{x}_k$ given the price. The price is then adjusted based on the gradient $h'(\nu)$, which is computed as:

$$ h'(\nu) = \tilde{y} - \sum_{k=1}^{n} \tilde{x}_k $$

where $\tilde{y}$ is the optimizer of $G(\delta - y) + \nu y$ and $\tilde{x}_k$ are the optimizers of the solvers’ subproblems. The process continues until the gradient is sufficiently close to zero, indicating that the optimal solution has been reached.

The mechanism elicits the solver’s true price for a given order size and terminates when the solver’s marginal price equals that of the CFMM. Instead of proposing an order size for each solver, the mechanism proposes one price, which it broadcasts to all solvers. The solvers respond to this price with a proposed order

The deterministic approach provides several advantages. It allows for the parameterization of utility and cost functions, making it suitable for empirical studies and model fitting with real-world data. The model allows for heterogeneity among solvers through individual cost and utility functions. The introduction of the aggregate forward exchange function $G$ provides a way to incorporate on-chain CFMMs into the model. The dual approach to solving the social welfare maximization problem naturally leads to a Dutch auction-like mechanism, providing a clear link between theory and practical implementation. Furthermore, this approach can be extended to more complex scenarios, such as multiple assets with correlated prices, by employing coordinate descent techniques. The deterministic framework complements the probabilistic model and offers a robust tool for analyzing and designing intent-based markets. The congestion extension demonstrates how the model can be adapted to account for more complex market dynamics, reinforcing findings from the probabilistic model.

Extension: Congestion

When the authors extend this to congestion they find that If solver cost functions are not independent and positively correlated, the user’s welfare decreases. This mirrors the probabilistic model’s result: under congestion costs, user welfare decreases.

In this work, we looked at an analysis of intent-based markets. The authors propose two models to model solver profiles and ultimately conclude that under certain conditions for user welfare, the designer of a market system might choose to limit participation. This can also be seen as the rise of oligopolies in intent-based market systems.

I recently read a quote by Einstein that says, “In theory, theory and practice are the same. In practice, they are not.” Having said this sometimes practical systems need theoretical groundings, there is enough evidence to see that different parts of solver-like markets in crypto are becoming oligopolies. Studying and modeling systems like this gives us a theoretical grounding to further explore other alternatives and motivates future improvements.

Off-Chain Compute Is All You Need

Mon, 27 Nov 2023 18:28:35 +0000

Towards a Proof-Based Future - Written with 0xkrane.

Introduction

Blockchains are globally distributed ledgers that come to a consensus over a global state. Some blockchains come equipped with a Turing-complete execution environment that enables programmability on top of this global state. Programs that target blockchains’ execution environments are called smart contracts, and the underlying blockchains are called smart contract platforms. Ethereum, Solana, and Avalanche are some of the most widely known smart contract platforms. We can think of smart contract platforms as distributed computers, with the execution environment (or virtual machine) acting like the CPU and the state performing the role of storage.

This framing of blockchains as computers will be important to motivate why coprocessors/off-chain compute is inevitable, especially in the context of blockchains. In traditional computing, coprocessors originated in microarchitecture to enhance performance. Similarly, coprocessors on Ethereum promise access to historical data and high-performant offchain compute to augment the features and design space of the base-layer protocol. Take a look at this introductory article on coprocessors for more.

This article explores coprocessors from first principles, aiming to clarify their importance and meta-properties. We then compare them to rollups, demonstrating how these two concepts, while different, are closely related. We also provide examples of when rollups and coprocessors can be used in conjunction with each other. For example, even an all-powerful rollup or L1 might need a coprocessor for heavy lifting tasks.

We conclude this article by observing that blockchains are moving towards a future where computation is centralized, but verification remains decentralized. Rollups, coprocessors, and any other form of verifiable off-chain compute are just different instantiations of this future.

How we got here:

In “The Limits to Blockchain Scalability,” Vitalik mentioned that for blockchain decentralization, it is important that regular users can run a node.

As previously mentioned, Ethereum can be conceptualized as a decentralized global computer in many aspects. It is a network of nodes running software that provides computational resources for executing smart contracts. The Ethereum blockchain stores state information and code, similar to a computer’s storage and memory. And the Ethereum Virtual Machine (EVM) runs on every node, processing transactions and executing code like a CPU. However, Ethereum is permissionless and decentralized, using consensus between untrusted nodes. If some nodes go offline, the network continues operating. To ensure the correctness of EVM operations, the validators on Proof-of-Stake (PoS) networks like Ethereum must perform all state transitions to verify them. This limits the speed of a PoS network to its slowest nodes, limiting the amount of computation app developers have available to them.

Unlike a regular computer, Ethereum limits computation and storage to prevent network abuse. Fees are charged for each operation, making endless loops financially impractical. This approach keeps barriers to entry low, allowing everyday hardware like a Raspberry Pi to run network nodes. The constraints enable an inclusive system where anyone can help operate the decentralized Ethereum network.

Due to these computational restrictions of Ethereum nodes, complex applications like Machine Learning models, games, or scientific computing applications cannot feasibly run directly on Ethereum today.

It’s a trade-off to make Ethereum widely accessible, secure, and sustainable as a foundation for basic apps. But inevitably, some limitations exist relative to a computationally unrestricted computer. It has limitations when compared to even an ancient processor like a Pentium 5:

No complex floating point math - The EVM only supports basic math and logical operations. Advanced numerical computations like neural networks are not feasible. (An interesting tidbit is an inability to handle floating point has also made swapping rebase assets like Ampleforth, etc, harder in recent history and sometimes even incompatible with some DEXs).
Limited computation per block - Gas fees meter computations, so complex software like games would be prohibitively expensive. The gas limit per block is 30M gas.
Restricted memory - Smart contracts have small permanent storage limits, making large programs difficult.
No persistent file storage - There is no way to store files like graphics, audio, or video on the blockchain.
Slow speed - Transaction speeds on Ethereum are currently ~15 TPS, many orders of magnitude slower than a CPU.

Ultimately, limited storage and compute restricts the degrees of freedom available to apps (these limits differ from blockchain to blockchain, but they always exist). People have compared blockchains to the compute-constrained environments of the 1970s-1980s, but we think there are some large differences between these two:

The growth of computing in the 1970s-1980s was rapid (with transistor count in microprocessors going from ~1,000 to ~1,000,000 during that period). But this growth didn’t mean people often bought or updated their computers. Since smart contract platforms are limited by their slowest nodes, a speed-up at the frontier of computers will not necessarily lead to blockchains seeing a proportional increase in computational speeds. A speed-up can only happen if the baseline requirements for nodes on the blockchain are updated.
There is also a clear trade-off between constantly updating the minimum hardware requirements for nodes and decentralization. Solo stakers might not want to upgrade hardware every couple of years (and they certainly don’t want to monitor performance daily), leading to only professionals wanting to run blockchain infrastructure.

All of this is to say that, over the years, CPUs improved, and we got more CPU cores on every device to allow us to do progressively complicated tasks. If we think blockchain computers won’t speed up as fast as traditional computing (due to baseline node requirements), then it makes sense to try to find alternative sources of compute. An interesting analogy to pull here is that CPUs in traditional computing didn’t get good at graphical processing tasks, leading to the rise of GPUs in almost every computer. Similarly, since blockchains are focusing on being secure stores of state with simple compute batteries enabled, there is a clear opportunity for off-chain compute to expand the application design space. Today, blockchains only make sense for low-compute applications that want properties like open access, self-sovereignty, censorship resistance, and composability. To put a greater variety of applications onchain, we need to lift the constraints we place on app developers. We say this with the understanding that these constraints have also been a boon for experimentation. For example, CLOBs couldn’t effectively run on Ethereum due to the compute constraints, so AMMs were adopted, having since clocked in a trillion dollars in volume.

There are two common approaches to making more compute available to blockchain applications:

Increase baseline node requirements relatively often. This is roughly the path integrated high-performance blockchains like Solana and Sui take. A high baseline for nodes makes it possible for them to build a very fast blockchain and also lifts some design constraints from application design. Phoenix, a Limit Order Book DEX on Solana, could not be built on Ethereum (or any L2) today. The flip side to increasing baseline requirements is that if they grow constantly, then running nodes might only be viable for professional infrastructure providers. Historical RAM requirements do a pretty good job of showcasing how hardware requirements have grown consistently on Solana:

Web Archive (Note: we use median RAM requirements from 2020)

Moving compute off-chain to third parties. This has been the strategy the Ethereum ecosystem has adopted. These third parties could themselves be blockchains (in the case of rollups), off-chain verifiable compute devices (i.e., coprocessors), or trusted third parties (as is the case with application-specific off-chain compute like dydx’s orderbook).

Towards Unification of Off-Chain Compute

Recently, there has been a rise in talks of coprocessors, which provide off-chain verifiable compute. Coprocessors can be implemented in various ways, including but not limited to Zero-Knowledge Proofs or Trusted Execution Environments (TEEs). Some examples are:

ZK coprocessors: Axiom, Risc Zero’s Bonsai.
TEEs: Marlin’s Oyster,

Simultaneously, when it comes to offloading compute, Ethereum’s rollup-centric roadmap offloads compute to various rollups that settle on Ethereum. Over the last couple of years, a steady stream of developers and users have been migrating to rollups due to a combination of cheaper, faster transactions and incentives provided by the rollups. In an ideal world, rollups allow Ethereum to scale its overall computational capacity via off-chain execution without adding trust assumptions. More compute doesn’t just refer to executing more transactions but also to doing more expressive computation per transaction. New transaction types expand the design space available to applications, and higher throughput reduces the cost of performing these expressive transactions, assuring affordable access to a higher class of applications.

Before we go further, let’s briefly define both rollups and coprocessors to prevent confusion:

Rollups: Rollups maintain a persistent, partitioned state different from their base/host chains but still inherits the security properties of their base by posting data/proofs to it. By moving the state off of the host chain, rollups can use additional compute to perform state transitions before posting proofs of integrity of these state transitions to the host. Rollups are most useful to users who don’t want to pay the high fees of Ethereum but want to access the security properties of Ethereum.

Before diving into coprocessors, let’s give some more background on how constrained smart contract development on Ethereum is today. Ethereum has persistent state storage in its global state - account balances, contract data, etc. This data persists on the blockchain indefinitely. However, there are limitations:

The maximum size of contract data is limited (e.g., 24KB per contract currently and was set in EIP 170). Storing large files would exceed this. (*Not solved by coprocessors either)

laurence @functi0nZer0

Alright, it's time to modify EIP-170 to raise the cap

Someone explain to me why we shouldn't beyond the risk of state bloat

Nov 17, 2023

Reading/writing contract storage is slower than a filesystem or database. Accessing 1KB of data can cost millions of gas.
While the global state persists, individual nodes only retain the recent state locally in “pruning” mode. The full state history requires an archive node.
There are no native filesystem primitives for handling files like images, audio, and documents. Smart contracts can only read/write basic data types to storage.

Solutions around this are:

Large files can be split into smaller pieces to fit within contract storage limits.
File references can be stored on-chain, with the files stored off-chain in systems like IPFS.

Coprocessors: Coprocessors don’t maintain any state themselves; they behave like lambda functions on AWS, where applications can send a compute task to them, and they send back the result with proof of computation. Coprocessors fundamentally increase how much compute is available to any given transaction, but since proving on coprocessors also happens on a per-transaction basis, using them is going to be more expensive than rollups. Given the cost, coprocessors are likely to be useful to protocols or users who want to do complex one-off tasks in a verifiable way. Another benefit of coprocessors is that they allow applications using off-chain compute to also access the full historic state of Ethereum without adding any trust assumptions to the application itself; this is not possible on a vanilla smart contract today.

To drive home the difference between rollups and coprocessors, let’s refer to the ZK flavors of both these primitives. ZK rollups access both the verifiability and the compression aspect of zero-knowledge proofs, allowing them to fundamentally increase throughput for their ecosystem. Coprocessors, on the other hand, only access the verifiability property of zk proofs, meaning the overall throughput of the system remains the same. Additionally, ZK rollups require circuits that can prove any program that targets the virtual machine for that rollup (for example, rollups on Ethereum have built zkEVMs for contracts that target the EVM). In contrast, ZK coprocessors only need to build circuits for the tasks they are enlisted to perform.

So, it looks like the two biggest differences between rollups and coprocessors are:

Rollups maintain a partitioned persistent state, and coprocessors do not (they use the state of the host chain).
Rollups (as the name suggests) batch several transactions together, and coprocessors are generally used for complicated tasks as part of a single transaction (at least in the current paradigm).

Recently, Booster Rollups have been proposed, which execute transactions as if they are running directly on the host chain, with access to the full state of the host. However, Booster Rollups also have their own storage, allowing them to scale computation and storage across both the host and the rollup. The Booster Rollup proposal points to how there is a spectrum in the off-chain compute design spectrum, with traditional rollups and coprocessors sitting on either end of this spectrum. Rollups, Booster Rollups, and Coprocessors all provide access to more compute and only differ in how much state they hold partitioned from their base L1.

In a talk at the Modular Summit, 2023 called “Shielded Transactions Are Rollups”, Henry De Valence talked about this exact concept and presented a very simple image to define a rollup:

The talk posits that any execution offloaded by the base chain to a third party is a rollup. Under his definition, coprocessors would also be rollups. This slightly differs from our view of unifying rollups and coprocessors under the banner of off-chain verifiable compute but the overall sentiment remains the same!

In his Endgame vision, Vitalik discusses a future where block production is centralized and block validation is trustless and highly decentralized. We believe this is roughly the correct model to think about what is happening now. In a zk-rollup, block production and state-transition computation are centralized. However, proofs enable verification to be cheap and decentralized. Similarly, a zk-coprocessor has no block production; it only accesses historical data and computes state transitions over this data. Computation on a zk-coprocessor is likely to always be performed on a centralized machine; still, the validity proof returned along with a result allows anyone to verify the results before using them. Maybe it is correct to restate Vitalik’s vision as: “a future where computation is centralized, but verification of centralized computation is trustless and highly decentralized.”

Same Same But Different

Despite their overall similarities, rollups and coprocessors serve very different markets today. One might ask, “If we can just use a coprocessor on ETH L1 and access its liquidity, why do we need rollups?” while this is a fair question, we think there are a few reasons why rollups still make sense (and present a much larger market opportunity than coprocessors today):

As previously mentioned, coprocessors allow you to access more compute in the same transaction than the L1. But they can’t help move the needle on how many transactions can be performed by the blockchain that is calling the coprocessor (if you’re thinking about batching, voilà, you’ve arrived at a rollup). By maintaining a partitioned persistent state, rollups can increase the number of transactions available to people who want to access blockspace with Ethereum’s security properties. This is possible because rollups only post to Ethereum every n blocks and don’t require all Ethereum validators to verify a state transition happened. Interested parties can just rely on the proof.
Even if you use coprocessors, you still have to pay the same order of magnitude of fees as any other transaction on the L1. On the other hand, rollups via batching can reduce costs by orders of magnitude.

Additionally, since rollups provide the ability to run transactions on this separate state, they still behave like blockchains (faster, less decentralized blockchains, but blockchains nonetheless), so they, too, have clear limits on how much compute can be accessed from the rollup itself. In this scenario, a coprocessor can be useful for rollups if a user wants to do arbitrarily complex transactions (and now you’re doing verifiable transactions on a rollup, so you only have to obey the laws of physics of the rollup).

Kobi Gurkan @kobigurk

👋 @Scroll_ZKP x @RiscZero 👋

in the spirit of “why can’t we all just along?” and composability - I wanted to run my bonsai zkcoprocessor on scroll!

(@__geometry__ portcos and two of my favorites)

as a recap, my coprocessor is performing many scalar mults on secp256r1

1/10 h

Sep 6, 2023

Another important point to note here is that most liquidity today resides on ETH L1, so for many protocols that rely on liquidity to improve their products, it might be astute to still launch on Ethereum mainnet. An application on Ethereum mainnet can get access to more compute by intermittently doing transactions on a coprocessor. For example, a DEX like Ambient or Uniswap v4 can use hooks in conjunction with coprocessors to do complicated logic on how to change fees or even modify the shape of the liquidity curve based on market data.

Tarun Chitra @tarunchitra

Rollups : Coprocessors :: imperative programming : functional programming

Kobi Gurkan @kobigurk

Let’s get some people angry today

Every rollup is a coprocessor

Sep 16, 2023

One interesting analogy compares the interplay between rollups and coprocessors to imperative and functional programming. Imperative programming focuses on mutable states and side effects, specifying step-by-step how to execute tasks. Functional programming emphasizes immutable data and pure functions, avoiding state changes and side effects. In the same way, rollups are like imperative programs that modify the state they hold, while coprocessors are like functional programs where they don’t mutate the state but produce a result along with proofs of computation. Moreover, just like imperative and function programming, rollups and coprocessors have their place and should be used accordingly.

A Proof-Based Future

If we end up in a world where computation is centralized, but verification of centralized compute is trustless and highly decentralized, where does that leave Ethereum? Will the world computer be reduced to a mere database? Is this a bad thing?

Ultimately, Ethereum’s goal is to give its users access to trustless compute and storage. In the past, the only way to access trustless compute on Ethereum was for computation to be performed and verified by all nodes. With the progression of proving techniques (especially zero-knowledge proofs), we can move much of the computation that happened on validator nodes to off-chain compute and only have validators verify the results on-chain. This essentially turns Ethereum into the world’s immutable bulletin board. The proofs of computation allow us to verify that a transaction was done correctly, and by posting them to Ethereum, we get a timestamp and an immutable historical store for these proofs. As zero-knowledge proofs become more efficient on arbitrary computation, it is likely that at some point the cost to do computation in ZK will be significantly less than the cost to do it on a blockchain (maybe even a 100-validator CometBFT chain). In such a world, it is hard to imagine that ZK proofs will not become the dominant mode of accessing trustless computing. Similar thoughts have been echoed by David Wong recently as well:

David Wong @cryptodavidw

so blockchains forced computation to be done by everyone, and zk allowed us to move some of that computation offline. How long til zk eats all the computation?

Nov 19, 2023

A future in which any computation can be proven also allows us to build infrastructure for the kinds of trustless applications that have user demand instead of trying to retrofit the Ethereum base layer to become the home for those applications. In the ideal case, tailored infrastructure will create more seamless user experiences and will also scale with the applications built on top of it. This will hopefully allow web3 applications to compete with their web2 counterparts and usher in the trustless, proof-based future cypherpunks always dreamt of.

All in all, we believe we are moving towards the following paradigm:

—————————Dont Trust, Verify—————————-

Blockchains: Unlocking a Privacy Layer for AI

Fri, 24 Nov 2023 21:18:58 +0000

The unprecedented rise of Large language models (LLMs) has seen a quick adoption of its technology in various spheres. While language models and transformers have been used before for Google Search, translation, and sentiment recognition, ChatGPT can be credited for bringing the transformer architecture and LLMs to the forefront of public attention and usage, seeing as many as 100 million users within a few days.

Due to their self-attention architecture, ease of training, and high parallelization capabilities, transformer models have taken the field of deep learning by storm. Their applications are prominent in Natural Language Processing, Computer Vision, Speech Recognition, Protein Folding, Reinforcement Learning, and other intersections of deep learning and sub-fields within artificial intelligence and beyond.

However, these models’ widespread adoption and eventual ubiquity raise critical questions about privacy and security. As these models interact with vast amounts of personal and sensitive data, safeguarding privacy without hindering functionality becomes paramount. Blockchains, a technology synonymous with security and decentralization, present a promising avenue to address these privacy challenges. By integrating cryptographic methods with the help of blockchains into AI inference and training, it could be possible to create a secure and transparent framework where data ownership and privacy are preserved.

This article explores one such method of convergence of privacy and AI, i.e., Multi-party computation in transformers. While implementations for MPC without the need for blockchains are available, Blockchains offer some guarantees to the framework, which aren’t possible in other such implementations. This convergence of blockchains and AI helps unlock a new privacy layer that balances the technological advancement of transformers with the ethical imperatives of modern society.

An abundance of AI applications

Chatbots

The most common way of interacting with LLMs today is Chatgpt, Claude, or one of the several such offerings. The reason for their popularity? The ability to draw on the trillions of words they are trained on, answer questions, give analogies to understand difficult concepts, and even do creative things.

While the following poem and sonnet are perhaps very cringe renditions, they offer a glimpse into their abilities, the ability to have a template or even a draft of anything you want at your fingertips.

We are still only in the early days of what is possible, though very long conversations are still impossible, and every conversation is essentially started anew. There is no notion of memory or remembering you. At some point, coherence also starts taking a hit when the model can no longer remember some of the earlier parts of the conversation or take some reference from there.

Programming

Programming is one of the biggest areas that has seen some acceptance and influence with LLMs.

Copilot, introduced by GitHub, offers coding services directly in your website IDE and offers it through extensions on Visual code as well. Several other IDEs have popped up that seem to have proprietary prompts tailoring LLMs for programming-specific tasks, offering it with GPT-4, which seems to be giving better results than the former.

Research has prodded along, simultaneously posting gains and improving performance as we learn more about the capabilities of the transformers. For example, the following work improved on prompting and showed the model’s ability to correct its wrong outputs through 3 methods:

Simple Feedback: Where you ask the model if it thinks a generated solution is correct or not
Unit Test: Where the feedback from unit tests is given to the model so it can improve upon its code
Code explanation: Where the model is asked to explain the code it has just written

All these approaches show that a model with these simple prompt upgrades outperforms works prior to it.

From “Teaching Large Language Models to Self Debug”

Where does this fail, though? The inability to assimilate large bodies of code is a current drawback for these models. Some possible use cases in that scenario are possibly writing higher-level tests after unit tests for certain functions are written, or helping debug large code bases where substantial effort is spent today.

Law & Medicine

Recently, one thing that has become popular when evaluating LLMs is reporting their ability to solve exams.

A quick look at GPT-4’s launch shows the results in popular exams being shown early on in the blog to establish model improvement over previous iterations of the model on a wide variety of different exams covering Law, Medical, Code, Aptitude, College entrance exams, AP exams, etc. While the importance of these can be questioned also whether they are good signals, a good natural question to ask is, “Where are we going”?

From https://openai.com/research/gpt-4

Using the following prompt

GPT-4 was shown to be able to explain Legal concepts to a very good degree

From “Explaining Legal Concepts with Augmented Large Language Models (GPT-4)”

Moreover, another work showed GPT-4’s ability to pass the Bar exam scoring 75%, which is enough to put it in the 90th percentile.

LLMs have also been shown to help with Medical challenges, performing well on various questions in the USMLE(United States Medical Licensing Examination).

From “Capabilities of GPT-4 on Medical Challenge Problems”

These abilities were shown to establish the idea that LLMs are quickly getting good at doing things, and while they aren’t perfect, they might be able to save a lot of time. Will they be at a place where they completely replace humans? Probably not until two key questions are answered:

In most law applications, Laws and precedents change over time, and large language models are trained on specific instances of data before some cutoff period; unless models are trained to update their internal knowledge, using these models for law applications ubiquitously is extremely difficult.
In most medical applications, when trained on datasets, LLMs and neural networks at large have been shown to be prone to biases and hallucinations. Medicine is a safety-critical application where the necessity to reduce false positives is key, and no model should prescribe the wrong medicine to patients.

Maths

Not only in direct user-facing applications like Medicine and Law, LLMs have also shown promise in their ability to reason and solve maths problems.

From “Solving Quantitative Reasoning Problems with Language Models”

While there’s a long way to go, and LLMs really lack in their ability to reason, lagging behind human performance by quite a margin, we could expect them to slowly start performing and getting well at these tasks as well. Terrence Tao, in his Microsoft blog, “Embracing change resetting expectations,” wrote:

With effort, human experts can modify ideas that do not work as presented into a correct and original argument. The 2023-level AI can already generate suggestive hints and promising leads to a working mathematician and participate actively in the decision-making process. When integrated with tools such as formal proof verifiers, internet search, and symbolic math packages, I expect, say, 2026-level AI, when used properly, will be a trustworthy co-author in mathematical research, and in many other fields as well.

The privacy question for AI

LLMs today must be deployed most efficiently and have some ways to go before they are fine-tuned for specific downstream tasks that help humans. Regardless of model performance, the way forward seems to be human collaboration and using these models with human feedback.

While the estimates for AI performance at superhuman levels might be off by a few years or even decades, LLMs have been slowly improving on every task broadly and have gotten much better since just a few years ago. These performance improvements suggest that LLM deployment in various places in the next few years might be inevitable.

In most companies, AI models served by corporations cannot be readily used for internal purposes because model serving occurs through APIs, and there is no guarantee that data sent today will not be added to training data for tomorrow’s LLMs. At the same time, the best possible use case for most companies in employing LLMs to properly use is to embed them along their codebases/documentation to help teams and increase productivity. Even at the more personal level, one possible use case of LLMs is their application in Medicine. Even if not to prescribe Medicine, they could be used to gather medical history or patient data before the doctor’s meetups. This prior interaction would expose the model to sensitive patient data, which might not be desired. Writing legal documents/researching cases within a firm could also reveal their internal files to a model where data privacy is paramount. When applied in research alongside researchers, while collaborative, researchers might want to silo their data and use the model for their specific interests.

Most personal use cases of AI, like chatbots, might reveal the data/preferences of the user over time, even if unintentionally. As chatbot APIs become stateful, they will remember user conversations and about the users themselves, which might provide overall QOL (quality of life) benefits but leak user data to the model and the company.

As LLM offerings proliferate and become ubiquitous either in the background of the apps you use or the interface with which you surf the internet or get things done online, private inference and private training of these models on your data becomes a significant problem to solve. This is not only limited to individuals but to those offering these solutions to entities who want to use it in-house on their dataset as well.

Another angle to look at is as AI vendors grow and offer solutions to most entities worldwide. The entity might want to test how good the model is on the hospital’s data, but this is sensitive private data or just IP, and the entity intends to keep it confidential from the AI vendor. This application is another such use case for private AI inference.

One final use case is when two different entities, in this case, hospitals, want to train a model together on private patient data; how do you offer services that keep the data private from each other but train the model efficiently?

While blockchains today are transparent ledgers of individual transactions using apps on different chains, they offer ways of coordination of decentralized networks and entities and also help in providing provable privacy guarantees for various use cases in tandem with other cryptographic methods like Multi-party computation, Zero-knowledge proofs, etc.

Multi-Party Computation on Blockchains

In Andrew Millers’ talk “MPC as a blockchain confidentiality layer”

Secure Multi-Party Computation (MPC) is a cryptography area concerned with creating methods that allow several parties to jointly compute a function using their private inputs without revealing them to each other. It enables collaboration on sensitive data, such as calculating a shared average without exposing individual numbers, thereby maintaining privacy for all participants.

Imagine three friends who want to know the average of their salaries without revealing their salaries to each other. They could use Secure MPC to accomplish this.

Assume the friends are Alice, Bob, and Eve:

Alice takes her salary, adds a random number to it, and tells the result to Bob.
Bob adds his salary and another random number to the number he received from Alice, then tells the result to Eve.
Eve adds her salary to the number from Bob, then subtracts all the random numbers added earlier and divides the result by three to find the average.

The final number is the average salaries; no one has learned anything about the others’ salaries. One nuance to pay attention to here is that although nobody knows the exact salary of each other if the average is lower than that Eve’s salary, then Eve could infer that one of the other two’s salaries is lesser than hers.

The above explanation is a very simplistic explanation of how it might occur, and we have yet to gloss over the technicalities of making this practical. Now that we have established some understanding of MPC, where do blockchains come in?

Blockchains, as we are familiar with them, are decentralized digital ledgers that have to ensure continuous operation (liveness) and resilience to failures or malicious behavior (fault tolerance). Liveness ensures that valid transactions are consistently added to the chain without delay. At the same time, fault tolerance allows the system to function correctly even if some network nodes fail or act dishonestly. Together, these properties create a trustworthy and robust system capable of handling various challenges without compromising the integrity of the recorded data.

Real-world applications of blockchains have slowly been growing and achieving escape velocity because they offer access to censorship-resistant, live ledgers. In the work “Credible, Optimal Auctions via Blockchains,” the authors explore the auction trilemma and show that when cryptographic commitments are used with the help of a censorship-resistant ledger, auctions are credible.

In “HoneyBadgerMPC and AsynchroMix: Practical Asynchronous MPC and its Application to Anonymous Communication,” the authors note that while MPC systems implementations exist in the wild, they do not offer any robustness or fairness guarantees, even though there have been other discussions and explorations of the problem none of them were practically implementations. In SMPC(Secure Multi-party computation), fault tolerance and liveness issues are just assumed and not explicitly paid attention to. Most blockchains today have not come up with ways of securing confidential data or have any confidential layers and offer zero privacy naturally. Blockchain with MPC provides a robust, secure method for the computation of private data with liveness guarantees even in the presence of possible adversarial or malicious entities. This complementary nature makes the infrastructure of blockchains synergize with the ideas that MPC offers.

While the entire treatment of how the exact implementation of MPC with blockchains would work is beyond the scope of this blog, you can look at these talks by Andrew Miller and Yunqi Li.

Note: You can skip to the next section if you are not interested in learning the mathematical technicalities of the transformer and how it is deployed using MPC.

MPC with Transformers

Now that we have recognized that MPC can be used with blockchains to provide privacy, fault tolerance, and robustness guarantees, we will now look at some methods for private inference on transformers with MPC.

The Transformer architecture

The transformer architecture

While going to every specific nuance of transformers will take a blog post in itself, I’d like to, in short, go through the architecture so we understand what makes transformers difficult when used in MPC settings.

When you are first given the input “The detective investigated,” the input is first tokenized, i.e., split into subwords and then substituted as numbers from a dictionary. This dictionary is also the vocabulary. The model’s vocabulary is built before the model is trained and can be thought of as similar to how humans have their own vocabulary. These are usually in the order of 250,000 words, but an adult has about a 30000 words functioning vocabulary, so models already operate at about 8-10x higher.

These are then represented as a vector, and the size of this embedding may vary according to the choices made during the architecture design. Once we have the input embedding as seen below, since transformer architectures have no notion of sequences and word order, we add positional encoding, i.e., in a very simple way, add some notion of word order so the model recognizes that “this is a ball” and “this ball is a” are different sentences and are to be treated somewhat differently.

Once the embeddings are obtained, they are sent through an attention layer, which can basically be thought of as a way to understand the language better and determine the relations between words. After which, it is sent through an MLP feed-forward neural network which lets the model learn about each word in reference to itself. While this is just one layer, in the original photo above, we see that there’s an “Nx” on each side of the architecture, which says that each of these modules is ultimately repeated $N$ times which determines the size and parameter of the model. GPT-3 essentially has 96 layers.

One intuitive way to think of what happens at higher layers is thinking that transformers learn more abstract representations at the later layers in the model. In comparison, the earlier layers make sense of the language itself.

How are these actually calculated?

Given an input vector $x$ and $(q, k, v)$ are input to the attention module, in this case, we assume only a single input vector $x$. Still, during actual computation, these are stacked together as matrices, and then a dot product is computed, after which it is put through a softmax function.

$$\alpha_t = \frac{\exp\left(\boldsymbol{q}^\top \boldsymbol{k}_t / \sqrt{d_{\text{attn}}}\right)}{\sum_u \exp\left(\boldsymbol{q}^\top \boldsymbol{k}_u / \sqrt{d_{\text{attn}}}\right)}$$

This calculates the “attention” in an input and

$$\tilde{\boldsymbol{v}} = \sum_{t=1}^{T} \alpha_t \boldsymbol{v}_t$$

this calculates the final output.

The usual way of representing this in most popular literature is

$$A(Q, K, V) = \operatorname{softmax}\left(\frac{QK^\top}{\sqrt{d_k}}\right)V$$

where the softmax function is given by

Softmax function

The softmax function is a nonlinear function that normalizes the sum of the inputs to 1.

The following formula calculates the MLP layer

$$\text{FFN}(\mathrm{x}) = \text{ReLU}(W_1\mathrm{x} + b_1)W_2 + b_2$$

These days GELU is used instead of the RELU function. The form of the function is as follows.

Graphs for RELU, GELU, and ELU

While we have mostly gone only over the separate modules of the architecture, something to observe is that towards the end, we have a linear layer which is then sent through the softmax, ultimately yielding probabilities over the vocabulary, and the highest probability word is chosen.

Why did we go over the transformer architecture? In general, when applied to Machine learning models, MPC has its own issues. Most Machine learning and Deep learning computation are done in floating point arithmetic, while MPC computation uses integers, and floating point emulation, even if possible, is very costly. Another reason is that most MPC applies basic arithmetic operations like addition and multiplication, and every other operation is composed or approximated with these. These simple operations make implementing complex functions costly. Finally, the issue we face with transformers is the presence of nonlinear functions like GELU and Softmax, which cannot be easily approximated.

Recent work like “MPCFormer: fast, performant and private Transformer inference with MPC” and “PUMA: Secure Inference of LLaMA-7B in Five Minutes” have offered some solutions to these problems. Both works provide approximations of these functions and address other issues that might arise because of these choices.

In MPCFormer: fast, performant, and private Transformer inference with MPC, the authors approximate the GELU function with the following, similar to how RELU is usually approximated.

$$GeLU(x) \approx 0.125x^2 + 0.25x + 0.5.$$

And the softmax approximation is made with the following quadratic equation.

$$\operatorname{softmax}(x) \approx (x+c)^2 \Big/ \sum (x+c)^2$$

The Gelu approximation is called Quad, and the Softmax approximation is 2Quad. The experiments find that Softmax numerically diverges from the original Softmax function (basically, they aren’t the same functions and have different graphs). Still, it is a good approximation that is computationally efficient.

The authors handle this specific divergence with the help of a technique called knowledge distillation, which was initially introduced in deep learning to use larger models as teachers to smaller models, so smaller models can learn the same things and have similar performance but be almost a magnitude smaller (sometimes half the size) than the larger model. In this case, this technique is used to make the model’s representations similar to the original model.

The model used in this work is the BERT model, which was the first pre-trained language model and preceded the GPT papers and work. The results in the work look promising in that they achieve a 5x speedup in the inference for a dataset called IMdB used for sentiment classification and GLUE, a natural understanding benchmark used for understanding and estimating model performance on language tasks. While IMDB has no actual loss in accuracy to provide MPC, i.e., privacy guarantees, we see that in the Quad+2Quad case, there is a 5-point reduction and just a 2x speedup compared to the original Bert implementation.

The PUMA: Secure Inference of LLaMA-7B in Five Minutes work improves the approximations for GELU, Softmax compared to the MPCformer work. The MPCFormer work also used some underlying assumptions in the MPC engine that caused the model to suffer in real-world performance, like not implementing some critical functions needed for transformer model inference. These are rectified in the PUMA work, which leads to better performance.

The GELU approximation in this work improves over the previous quadratic approximation in MPCFormer by studying the graph for the functions, as we saw earlier in the article. We see that after -4, the function tends to 0; after 3, it resolves to $x$ itself. The function between -4 and 3 is approximated with the help of Numpy.

And a better approximation is similarly found for softmax as well. In the case of BERT models, we see that the PUMA work gives almost a 2x inference boost, and the communication cost is practically reduced by 2.5x. The communication cost is the cost born by the MPC system to coordinate the computation that it does over function.

Time in Seconds and Communication in GB, PUMA results

Some future work to consider is that the PUMA work also experiments with models like LLama, a recent architecture trained by Meta, which shows much promise in the future of LLMs. Still, when applied in the MPC setting, they find that it takes about 500 seconds to generate one token, whereas, in real-world conditions without privacy, we see that LLama can generate about 20 tokens every second, which is almost a 1000x slowdown. We need to use better ways and use hardware acceleration techniques and quantization as well, which can speed up inference for MPC-based transformer inference.

Blockchains for Inference only?

One natural question is, “Are blockchains only useful for private inference of transformer models?” While the primary use case for AI x blockchains is a privacy layer, there are also some other aspects.

Will MPC be helpful for private model inference en masse? Blockchains make MPC protocols possible more securely and fairly but ultimately have limitations when implementing transformer inference protocols (training has not been explored yet). Ultimately, we approximate non-linear functions to make it possible within MPC, but while this is possible in an inference setting, it’s extremely difficult to train a neural network. Without its non-linearities, a neural network is simply a flat linear function. These approximations ensure decentralized inference, not training (yet). While not an optimized state-of-the-art deployment, the paper I covered takes about 5 minutes to generate one token from a 7B parameter Llama model. In contrast, you can generate 20 tokens locally on your CPU. The privacy vs efficiency tradeoff for deep learning is skewed, and we have a long way to go.

As Large language models become more capable, current evaluation methods of understanding their abilities become difficult. Evaluation datasets last for a short time, and just a few iterations of the model make the evaluation benchmark obsolete. The best method of changing and growing evaluation benchmarks for these models is to use expert evaluation. However, coordinating a body of experts and building an unbiased evaluation protocol is complicated. Blockchains offer a way for incentive alignment and bootstrapping expert networks in various fields and also design a protocol that might help expert evaluation of LLMs and the growing AI capabilities in a scalable and transparent way.

One last application to mention is this recent work called “Tools for verifying neural models training Data,” which introduces a concept called “Proof-of-Training-Data,” which is any protocol that allows a model trainer to convince a Verifier of the training data that produced a set of model weights. As model training takes trillions of words, some data might be private or copyrighted, and there’s a need to establish if a model used a specific trough of data. When adopted in the broader framework of zero knowledge, protocols like these offer a way of trustlessly establishing data provenance.

Conclusion

This blog is a soft introduction to how blockchains could act as a privacy layer for AI models and provide private inference for data-sensitive applications. In subsequent works, we could probably explore MPC protocol implementations in technical detail through blockchains, the tradeoffs, and design decisions that we need to establish, and if we can also introduce Zero-knowledge proofs with MPC to provide other guarantees that might be desired in such a system.

It offers an inside view into the world of large language models and some applications that are slowly making their way into the hands of millions of people. But with this widespread access, issues like data privacy become evident. We then discuss the various privacy pitfalls of LLMs. MPC with blockchains offers a way to do computation while keeping the data private with robustness and fairness guarantees. We finally look at recent works that apply MPC to transformer inference and discuss future work.

I will cover ZK-ML and FHE-based deep learning protocols in future articles. Thanks for reading!

A Brief Intro to Coprocessors

Fri, 06 Oct 2023 15:27:35 +0000

Towards unlocking a new class of applications. Not the compute we need, but the compute we deserve.

Decentralized apps face limitations in performing complex on-chain computations due to Ethereum’s restricted processing capabilities. As a result, we’ve seen many DeFi protocols move components like order books and risk systems off-chain. This points to a need for customized computational environments tailored to specific tasks.

We’ve seen a slow but gradual shift of many defi apps deployed onchain managing parts of their system off-chain. Dydx V4 is going to keep its order book off-chain and possibly its margining system as well. Blur keeps parts of its exchange off-chain for smooth UX. Aevo, an options exchange, keeps its order book and risk engine off-chain. The simplest reason for this is the difficulty of maintaining these performance-centric systems on-chain efficiently and in a scalable manner.

The migration of components off-chain points to a broader need - customized (and performant) computational environments tailored to specific tasks. This is not all, though. In this regime, the status quo works well. When a protocol runs an off-chain system, it’s ultimately opaque to you, the user, on trusting if the off-chain system works as it said it does. Verifiable computation does away with trust assumptions, allowing protocols to do off-chain computation without introducing trust factors. This is the promise of coprocessors for Ethereum. Before discussing the coprocessor model in Ethereum, let’s briefly recap where this idea stems from.

The concept of coprocessors originated in computer architecture as a technique to enhance performance. Traditional computers rely on a single central processing unit (CPU) to handle all computations. However, the CPU became overloaded as workloads grew more complex.

Coprocessors were introduced to help – specialized processors dedicated to particular tasks. For example, graphics processing units (GPUs) handle the immense parallel computations needed for 3D rendering. This allows the main CPU to focus on general-purpose processing. Other common coprocessors include cryptographic accelerators for encryption/decryption, signal processors for multimedia, and math coprocessors for scientific computations. Each coprocessor has a streamlined architecture to perform its niche workload efficiently. (Although you could say most of this has been subsumed by parallel programming, ala GPUs.)

This division of labor between CPU and coprocessors led to orders-of-magnitude improvements in performance. The coprocessor model enabled computers to take on increasingly sophisticated workloads not feasible with a lone generalist CPU.

Ethereum can also be considered a generalist CPU VM and is not equipped to do heavy computations simply because of the barring costs that one would have to pay for it to run on-chain, something that has constrained the deployment of a variety of protocols, even forcing designers to come up with something new within the constraints of the EVM. Put simply, costs are too restrictive for complex applications. This has also led to various protocols keeping parts of their protocol off-chain, and every off-chain model thus deployed has brought along with it a certain notion of risk. A risk of centralization and a risk simply of trust; you trust the protocol not to be malicious, which is somewhat against the ethos of decentralized apps.

In this article, I try to look at a few of these solutions and offer a glimpse into what kind of applications could be unlocked by virtue of this infrastructure. I will also try and look into alternative ways of offloading computation, which is sure to become a cornerstone of applications within the crypto space.

Zk-coprocessors

Coprocessors like those offered by Axiom and RiscZero have recently opened up a new dimension of applications possible on-chain by allowing smart contracts to offload heavy computation. The systems offer proof that the code was executed in a way anyone can verify.

Bonsai and Axiom are similar solutions in that they allow arbitrary computation with access to the on-chain state to be run off-chain and provide “receipts” that the computation was performed.

Axiom

Axiom enables Ethereum smart contracts to access more historical on-chain data and perform complex computations while maintaining the decentralization and security of the network. Currently, contracts have access to very limited data from the current block, which restricts the types of applications that can be built. At the same time, allowing contracts to access the full historical archive data would require all network nodes to store the full archive, which is infeasible due to storage costs and would negatively impact decentralization.

To solve this problem, Axiom is developing a “ZK co-processor” system. It allows contracts to query historical blockchain data and perform computations off-chain via the Axiom network. Axiom nodes access the requested on-chain data and perform the specified computation. The key is generating a zero-knowledge proof that the result was computed correctly from valid on-chain data. This proof is verified on-chain, ensuring the result can be trusted by contracts.

This approach allows contracts access to far more data from chain history and the ability to perform complex computations on it without burdening the base layer nodes. Axiom believes this will enable new categories of applications that rely on provable, objective analysis of historical blockchain activity. They have already launched mainnet functionality for basic data reads and plan to expand to full archive data access and ZK verification of contract view functions in the near future. Their longer-term vision is even more advanced ZK computations beyond EVM capabilities.

By generating proofs of correct off-chain execution, Axiom unlocks new categories of blockchain applications.

Risc Zero Bonsai

Risc Zero has developed a general-purpose zero-knowledge virtual machine (zkVM) that allows proving arbitrary programs written in languages like Rust, C/C++ and Go in zero knowledge.

The zkVM allows developers to prove arbitrary Rust code in zero knowledge without needing to design custom circuits. The goal is to make zero-knowledge application development more accessible. The zkVM generates a proof receipt that attests the program was executed correctly without revealing private inputs or logic. This allows intensive computations to happen off-chain, with the proof receipts validating correct execution on-chain. Rust crates work in this zkVM, but there are some limitations around system calls. A feature called continuations allows splitting large computations into segments that can be proven independently. This enables parallel proving, thus removing limits on computation size, and allows pausing/resuming zkVM programs. Continuations have enabled new use cases like fully homomorphic encryption, EVM, and WASM in the zkVM.

Bonsai is an off-chain zero-knowledge proving service developed by Risc Zero to enable the use of their general-purpose zkVM for Ethereum and blockchain applications. It provides a bridge between on-chain smart contracts and off-chain computations in zkVM.

The workflow enabled by Bonsai is as follows:

The developer writes a smart contract that calls out to Bonsai’s relay contract to request an off-chain computation
Bonsai watches for these on-chain requests and executes the corresponding zkVM program written in Rust
The zkVM program runs in Bonsai’s infrastructure, performing the intensive or private computation off-chain, and then generates proof that it was executed correctly.
The proof results, called “receipts,” are posted back on-chain by Bonsai via the relay contract.
The developer’s smart contract receives the results in a callback function

This allows computationally intensive or sensitive logic to happen off-chain while still validating correct execution via zk proofs on-chain. The smart contract only needs to handle requesting computations and consuming the results.

Bonsai abstracts away the complexity of compiling Rust code to zkVM bytecode, uploading programs, executing in the VM, and returning proofs. Developers can focus on writing their program logic. This infrastructure thus enables running general-purpose computations off-chain while keeping sensitive data and logic private.

Bonsai enables developers to build blockchain applications with off-chain computing straightforwardly without needing expertise in the underlying zkVM cryptography and infrastructure. Simply put, Bonsai enables developers to integrate off-chain computations easily without zk expertise.

Alternative Solutions

Is a ZK-coprocessor the only way to achieve verifiable off-chain computation? What other applications exist to offload computation in a trustless and secure way? While opinions about the security properties, efficiency, and implementation differ, they are being explored in various corners of crypto and will come to the forefront slowly.

Alternatives like MPC and TEEs provide other approaches to verifiable off-chain computation. MPC allows joint computing on sensitive data, while TEEs offer hardware-based secure enclaves. Both present tradeoffs but can be alternatives for ZK-coprocessors.

MPC (Multi-Party Computation)

Secure multi-party computation (MPC) allows multiple parties to jointly compute a function over their inputs while keeping those inputs private. It enables collaboration on sensitive data, thereby maintaining privacy for all participants. However, achieving fairness in MPC, where either all parties learn the output or none do, is impossible if most parties are dishonest. In other words, privacy and integrity guarantees disappear when all nodes are corrupted. Blockchain technology can help make MPC protocols fairer.

Imagine three friends who want to know the average of their salaries without revealing their salaries to each other. They could use Secure MPC to accomplish this.

Assume the friends are Alice, Bob, and Eve:

Alice takes her salary, adds a random number to it, and tells the result to Bob.
Bob adds his salary and another random number to the number he received from Alice, then tells the result to Eve.
Eve adds her salary to the number from Bob, then subtracts all the random numbers added earlier and divides the result by three to find the average.

The final number is the average of their salaries; no one has learned anything about the others’ salaries. One nuance to pay attention to here is that although nobody knows the exact salary of each other if the average is lower than Eve’s salary, then Eve could infer that one of the other two’s salaries is lesser than hers.

The blockchain provides a tamper-proof public ledger that allows parties to post information. By using witness encryption, parties can encrypt the output of an unfair MPC protocol. They post tokens to the ledger that allow extracting a decryption key. Since the ledger is public, all parties can access the decryption key at the same time. This enables a fair decryption protocol where either all parties decrypt the output, or none do.

In “MPC as a Blockchain Confidentiality layer,” Andrew Miller talks about how MPC could help computations on private data. Publicly auditable MPC uses zero-knowledge proofs to retain integrity despite total node corruption. Clients would commit inputs to prove validity. The MPC nodes generate proofs of correct computation. Ultimately, the verifiers will check inputs, outputs, and proofs match. This auditing incurs minimal overhead beyond standard MPC costs. Proofs are succinct using SNARKs with a universal setup. However, questions remain about performance optimizations, programming models, and real-world deployment.

Secure Enclaves / TEEs

Sensitive data like personal information, financial data, etc., must be protected when stored or in transit and when it is being used and computed. Traditional encryption methods protect data at rest and in transit but not when data is actively being used. This is a problem because when data is being processed, it is often in an unencrypted form, leaving it vulnerable to attack.

Trusted execution environments (Or secure enclaves) are isolated environments where data can be encrypted, but computations can still be performed on them. The key idea is to isolate the data and computation so even privileged system processes can’t access it. Trusted Execution Environments (TEEs) are secure areas within a processor that provide hardware-based security features to protect sensitive data and code. They isolate specific software from the rest of the system, ensuring that the data within the TEE cannot be tampered with, even by the operating system or other software running on the device.

TEEs allow sensitive data to remain protected even while it is being used. This enables applications like privacy-preserving social networks, financial services, healthcare, etc. There are some limitations around efficiency and trust assumptions, but enclaves are a powerful technology with many potential uses, especially when combined with blockchain networks to build robust, uncensorable systems. The tradeoffs around trust may be acceptable for many commercial and non-profit applications where strong data privacy is required.

Trusted execution environments (TEEs) allow you to outsource computations to an untrusted third-party cloud provider while keeping your data confidential and operations tamper-proof. This is hugely useful for decentralized apps and organizations that want to take advantage of the convenience and cost savings of the cloud without sacrificing privacy or control. But TEEs don’t magically solve all problems - there are still some practical challenges to work through before most developers can easily use them.

For example, verifying that a TEE runs your expected code independently is hard. Subtle differences between builds mean that reproducing the exact same binary is tricky. This makes auditing difficult. Persistent storage is another issue - TEEs are isolated environments without permanent data storage. But real apps need to preserve state across reboots. This requires careful design to securely communicate data between the trusted TEE and the untrusted regular system. (Edited this mistake).

They are a powerful building block but still need thoughtful systems research to address their limitations around the one mentioned above and vendor centralization, scaling, and fault tolerance.

Trusted execution environments (TEEs) like Intel SGX and AWS Nitro Enclaves provide isolated environments for running sensitive computations and storing confidential data. TEEs ensure that even privileged system processes cannot access or tamper with code and data inside the TEE. This allows decentralized apps and organizations to outsource computations to untrusted third-party cloud hosts without worrying about privacy or integrity.

Solving these challenges will greatly expand the applicability of TEEs for decentralized apps needing strong integrity, confidentiality, and censorship resistance while outsourcing computation and storage to untrusted clouds. TEEs are a powerful primitive, but thoughtful system co-design remains necessary to address their limitations.

A brief comparison

When evaluating coprocessors, an important consideration is the security model and level of assurance needed for different types of computations. Certain sensitive calculations, like matching orders, require maximal security and minimal trust assumptions. For these, coprocessors using zero-knowledge proofs like zk-coprocessors provide strong guarantees, as results can be verified without trust in the operator.

However, zk-coprocessors might have downsides in efficiency and flexibility. Other approaches like MPC or trusted hardware may be acceptable tradeoffs for less sensitive computations like analytics or risk modeling. While providing weaker assurances, they enable a wider array of computations more efficiently. The level of security needed depends on the risk tolerance of applications. Teams should analyze the value at stake and evaluate if unverified but efficient coprocessors are a reasonable engineering compromise for certain non-critical computations.

Overall, coprocessors span a spectrum of security models, and teams should match solutions to the security requirements of specific tasks. The ecosystem is still nascent, so further advances in scalable verifiable computation will broaden the possibilities.

Applications

Dynamic Control for Lending Protocols

In the blog “Feedback Control as a new primitive for Defi,” the authors mention that control mechanisms for defi mechanism might slowly upgrade from one end to another, using reinforcement learning (RL) and DRL as computation and storage becomes abundant. While RL might still be difficult, Machine learning applications might still be possible due to verifiable computation.

Lending protocols in the past year have come under scrutiny because of the possibility of bad debt due to aggressive parameters for the token being lent in the otherwise liquidity-absent bear market. Models that can access on-chain liquidity and create a liquidity profile for assets could possibly dynamically change parameters.

For example, Lending protocols could benefit from dynamically controlling interest rates based on real-time on-chain data. Rather than relying on preset interest rate models, a feedback control system could adjust rates algorithmically based on current utilization and liquidity factors.

For example, when demand for borrowing an asset is high, pushing utilization rates up, the controller could rapidly increase interest rates to incentivize supply and stabilize utilization. Conversely, when utilization is low, rates could be reduced to incentivize borrowing. The controller parameters could be tuned to optimize for objectives like maximizing protocol revenue or minimizing volatility.

The protocol would need access to real-time on-chain data like total borrowed amounts, liquidity available, and other utilization metrics to implement this. The controller logic then processes this data to compute optimal interest rate adjustments. The rate updates could be governed on-chain via a DAO or off-chain with proof verification. Although recent work, “Attacks on Dynamic Defi Interest rate curves” by Chitra et al. has shown that dynamic lending models result in more MEV. So, the design of these protocols needs careful consideration.

ML applications

The abundance of access to blockchain data leads us to a natural conclusion of using machine learning applications this way. While proving computation for machine learning applications might be slightly harder, verifiable ML computation is a huge market on its own. These could also be utilized for on-chain applications, especially in some security applications.

Blockchain data contains valuable signals that machine learning models could use to detect suspicious activity or power risk management systems. However, running ML on-chain is currently infeasible due to gas costs and privacy concerns. This could look like on-chain Monitoring systems for smart contracts, wallets, or portfolio managers for detecting suspicious withdrawals or transfers. There is a vast amount of profiling data available for various kinds of signals to be obtained in the case of security, it would be for “Ruggers,” “Hacks,” and other malicious attacks. It can also be used for defi applications for creditworthiness and profiling risk for lenders and borrowers given their onchain history.

Challenges include data quality, concept drift, and performance limitations of proof systems. But by combining ML with verifiable off-chain computation, coprocessors open up many new opportunities for blockchain analytics and risk management.

Perpetual swaps and Options

Margin systems for perpetual swaps have always been hidden from users regarding centralized and even decentralized exchanges. Margin systems for derivatives like perpetual swaps and options have traditionally been opaque black boxes controlled by centralized exchanges.

Coprocessors present an opportunity to implement transparent and verifiable margining logic for decentralized trading. The promise of implementing auto-deleveraging systems in a verified way offers a higher trustworthiness factor for users and immediately differentiates them from their centralized counterparts.

The margining system could monitor indexed price feeds and position values for perpetual swaps, liquidating positions before their margin balance goes negative. All risk parameters like maintenance margin ratios, funding rates, and liquidation penalties could be governed on-chain.

However, the intensive computations for calculating margin balances, unrealized PnL, liquidation amounts, etc., can be offloaded to a coprocessor. It would execute the margin engine logic in a confidential environment and generate proofs attesting to correct computation.

The benefits of the coprocessor approach include transparency, verifiability, and privacy. Margin engine logic is not a proprietary black box anymore. Computations happen off-chain, but users can trust proofs of correct execution. The same could be achieved for options as well.

Challenges include efficiently generating proofs for intensive margin calculations. But overall, coprocessors unlock new potential for decentralized derivatives platforms by combining privacy with verifiability.

Conclusion

Coprocessors greatly expand the possibilities for blockchain applications without compromising decentralization. As cutting-edge projects continue innovating in this space, the future looks bright for verifiable off-chain computation on Ethereum and beyond.

In a future article, I will dive into these solutions’ security considerations, comparisons with rollups, how they fit into the broader ethereum application landscape, and if they are a panacea to scaling problems.

A List of Open Problems in Crypto - II

Wed, 04 Oct 2023 22:02:43 +0000

A reminder of progress and problems unsolved.

We’ve come a long way since the first time I wrote this list in Jan 2022. It has been inspiring to see the space proliferate and build extremely interesting infrastructure for the future of crypto, but we have a long way to go.

I believe that having an open list of problems is a good reminder of progress for a field, nascent or old, research problems or practical applications. These also provide a benchmark to look towards and glance upon while taking stock of the progress the field has made over the years.

While I wrote about open problems just in Defi last time, this time, I attempted to be a little ambitious and cover open problems in crypto. These aren’t just research problems; some will ultimately be protocol-level solutions, and some will even be applications.

Decentralized Finance
Mechanism Design
MEV
Zero Knowledge
Other problems in Crypto

Decentralized Finance [Defi]

Automated risk scoring of lending borrowing pools
- Possible Solution: Risk assessment without historical data is really hard because it could be so that a pool of users with a good credit history will always pay the loan back. “Credit score” in tradfi.
- How can we accurately model the risk of default for a pool of borrowers without relying on traditional credit data? → Answering this might lead to a framework for lending pool risk scores.
A dynamic Lending market parameter model
- Bad Loans have been increasing. Gauntlet/risk monitoring teams are doing well, but DAOs are moving slowly (Curve example).
- How do you solve for low liquidity coins? The model could be a function of liquidity and adjust rates dynamically for a given pool.
- Though there are downfalls to this model, in “Attacks on Dynamic Defi Interest rate curves,” Chitra et al. show that dynamic lending models have more MEV.
Managing Risk for lenders and distributing risk/ Undercollateralized Loans
- While active monitoring is excellent. How do we distribute this risk and make it efficient?
- Is there a world where we could have undercollateralized loans on-chain?
  - Are off-chain contracts the only way? (Wildcat protocol)
Private Lending
- Lending protocol transparency of pools has led to the hunting of liquidation levels, which have become Schelling points for traders.
- How do you design a privacy mechanism that reveals minimal information?
Designing Cross-chain Defi
- As lending markets become chain-centric, liquidity across assets is becoming fractionalized.
- As liquidity generally gets more fractionalized, how do you source liquidity seamlessly at size and settle it?
- Look at Cross-chain Margining Systems from The Defi Prime Broker for the DEX version.
- Defining the safety of these protocols in terms of balance between synthetic assets (cross-chain) and Native assets (on-chain)
What does the future of Spot Dexes look like?
- Can CLOBs be designed to accommodate tail assets?
  - Is there a unification of AMM/CLOBs beyond CLAMMs possible?
  - CLOBs historically haven’t worked for tail assets (Etherdelta was bad)
LP Profitability Problem: What is the optimal strategy for passive and active liquidity providers?
- How can you formulate and model the problem? → Possibility that it doesn’t exist
  - Does Uniswap-X RFQ mean this is not possible onchain? Onchain LPs in uniswap X have become LPs of last resort and get toxic flow possibly.
  - How do you protect LPs better?
How does DEX design solve for LVR (Loss-Versus-rebalancing)
- Dynamic fees - How do we best set fees dynamically based on volatility and other signals to optimize returns?
- Orderflow Discrimination - If you know how to discriminate between uninformed and toxic flow, you could possibly charge them different fees, etc

Open Research Problems - Mechanism Design

Restaking Equilibria
- Competitive Equilibria Between staking and on-chain lending discusses how lending and staking equilibria can exist.
- What does this look like in a restaking world where the return from staking isn’t exactly uniform but definitively higher than simple staking?
- What does restaking mean for the security of the base layer? Is restaking an anemic phenomenon to the security of a base layer?
The privacy-information tradeoff for DEXs (privacy-efficiency frontier) - No Free Lunch theorem
- In Differential privacy in CFMMs, authors show that partial privacy in CFMMs is possible and that there is a tradeoff between price (execution) and privacy. Private Dex Architectures have been rising recently. How do they address this?
The privacy-information tradeoff for MEV (privacy-efficiency frontier) - No Free Lunch theorem
- The privacy-efficiency tradeoff in sharing MEV information. Privacy is needed to decentralize the MEV supply chain, but more privacy usually means less efficiency.
Mechanism Design (Private & Verifiable)
- Adding ZK to existing mechanisms like CFMMs and auctions does not automatically guarantee strong privacy.
- Verifiable and auditable mechanisms for applications like auctions, order flow, and matching markets
- Adopting Mechanisms for a ZK World
Designing multi-resource Fee markets
- Resources: Dynamic Pricing for Non-fungible Resources and Transaction fee mechanism design
- Blockchain resources are being focused on at different granularities (blob market introduced to handle ephemeral data). Does it still make sense to meter all resources at the same level?
- Finding the right level of pricing granularity between opcodes and full applications to optimize productive efficiency and therefore, develop robust local fee market designs that can practically segment demand and allocate execution efficiently.
- Multidimensional EIP 1559 Model from Vitalik

Open Problems in MEV

Minimizing latency advantages in MEV/ Geographic Decentralization
- Talks by Phil Daian and Robert Miller
- Low latency provides advantages in optimizing and extracting MEV. This could incentivize geographic centralization if MEV parties co-locate to minimize latency.
- The challenge is designing MEV systems that are not sensitive to latency and allow geographic distribution of nodes.
- Ideas include allowing deferred transaction specification at block-building time rather than sending individual transactions.
- Also, look at “Exploration of MEV Latencies”.
Credible Private Auctions on-chain
- In an everything-is-an-auction world, how do we hold auctioneers accountable and trustable?
  - Chitra et al. show a possible world. When will we get this onchain?
  - What are other guarantees that we need for auctions to be practical?
Design of Order Flow Auctions
- OFAs follow a common framework with four components: originators/orders, auction/info sharing, bidders/bids, and winning bid/inclusion.
- Key design decisions exist around order types, information sharing, bidder permissions, bid selection, and execution guarantees.
- More at “The Orderflow Auction Design Space”
MEV Distribution Applications
- MEV is not evil or has any nature associated with it, just an emergent property of an economic system (Can be thought of as inefficiencies being captured)
- If MEV is captured by the applications and redistributed in the case of AMMs, in essence, LPs would get value or users in the case of sandwiching.
MEV Mitigation
- Recent work has shown private RPCs don’t prevent users from experiencing slippage (a popular belief among private RPC users). Are commitments necessary for MEV mitigation?
- How do you design UX to improve this at the user end?
A framework for Sequencing rules given payoff/Application?
- Chitra et al.’s Theory of MEV II proves that MEV handling should be application-specific/ Sequencing criteria for payoffs (properties proven in the paper)
- Better lower bounds for specific applications/payoffs when sequencing is defined (already shown for CFMMs, $O(\log n)$ when abundant liquidity)
Generalizing PBS
- How to generalize PBS (proposer-builder separation) to support more flexibility like partial blocks, different block specifications, inclusion lists, etc.
- Also relevant in the L2 world
  - A key problem is whether to enshrine MEV auctions in the L2 protocol, burn MEV to incentivize proof production, or leave it to proposers and builders. More research is needed on the economics.
  - Other open problems include determining optimal mechanisms for proposer decentralization, managing high compute needs for L2 block production, prover incentivization, and enabling permissionless participation.
Censorship Resistance Mechanisms
- Today, five out of the six largest block builders comply with the OFAC sanctions.
- How do you design mechanisms for avoiding this?
- To learn more, listen to this talk here.

censorship.pics - 60% of slots are censored

Some Problems to Upgrade Zero Knowledge

Impossibility results for Zero-Knowledge Crossovers like ML and Defi
- Are some applications that are not possible or bounded for some and prevent practical usage?
Separability results for Zero-knowledge Crossovers
- Are there any results that possibly allow you to say with ZK, you get better efficiency properties for payoffs, and without ZK, they don’t?
The Imagenet of ZK - Towards a ZK benchmark
- Imagenet competition ultimately brought about the revolution in AI in the form of deep neural networks. Alex Krizhevsky wrote custom kernels for training Alexnet, and thus GPUs started getting adopted for Deep learning.
- Is there a ZK equivalent benchmark to make ZKs more performant?
  - Opening this up as a yearly benchmark incentivizes research groups to work on this.
- The imagenet of ZK could possibly do the same for ZK (maybe need a yearly competition)
Torch/Tensorflow for ZK circuits
- Zero Knowledge feels like it’s in the Cuda era
- Computational graph models for circuits are similar to how neural networks are written.
- There is no standard framework for ZK applications
A Fast.ai for Zero Knowledge
- Some credit to Deep learning’s growth was the ability to top-down learn and this course design subsequently.
- What do toy applications for learning ZK look like?
Zero Knowledge Identity/ ZK-KYC
- Real-world application with outsized impact
- It does not have to be strictly KYC-centric
- Issues around making it government-compliant
ZK Deep Learning/ZKML
- Verifiable computation inference?
  - Explored for some models in the Cost of Intelligence: Proving Machine Learning Inference with Zero Knowledge by Modulus Labs.
  - Does this need the existence of ZK provers for Models specifically?
- Is Zero knowledge Deep learning possible?
- Some other ZK applications possible - Data/Training Provenance
  - Tools for Verifying Neural Models Training Data Any such addition to a system would help determine compliance issues/or verify that the model was actually trained on the data.
  - Also explored in “Experimenting with Zero-Knowledge Proofs of Training.”

An Incomplete Primer on Intents

Wed, 27 Sep 2023 22:19:43 +0000

Getting to know about an ~~intense~~ intent-centric world & some other thoughts.

Intents have been slowly getting popular in the research discourse in the crypto space, and the concept is being built upon by various protocols like Anoma, Essential, etc.

This article is meant to be a primer to the various views, ending with what an intent resolution architecture could look like where intents are expressed in natural language. If successful, intents show the promise of revolutionizing application architecture at every level. Hence, the concept merits discussion, delving into its nuances and what it might take to reach there.

Understanding an Intent

Intents allow users to specify certain transaction conditions or preferences without providing exact message calls. This allows for more flexibility and potentially less on-chain complexity.

In “Intent-based architectures and their risks,” the definition is, “an intent is a set of declarative constraints which allow a user to outsource transaction creation to a third party without relinquishing full control to the transacting party.”

In a recent podcast, Chris Goes from Anoma defines it in two ways: “Credible commitments to preferences over a state of some system” and “credible commitments to information flow constraints.”

An intuitive way of thinking about intents is that intents are basically, Intents are desired outcomes. When you express an intent, you simply define the outcome you want, not the process to get there.

Let’s say your desired outcome is trading some Tether (USDT) for Ethereum (ETH). Instead of managing the whole process yourself - picking the exchange, making accounts/signing transactions, handling transfers (or converting dust in your wallet), etc. - you submit an intent stating:

“I want the outcome of trading my 1 Ethereum for 2000 USDT.”

Some other entity, called a Solver, takes your intent and figures out how to fulfill it. The solver handles the messy details of trying to optimize for the best possible outcome for you.

The key is that intents focus on outcomes rather than processes. You define the desired results, while someone else determines the “how.” Intents greatly simplify the transaction flow that most users use in crypto by allowing you to specify outcomes and not worry about steps.

Now that we have built the basic idea, the higher-level idea is that users define what they want without specifying the contracts they want to route their transactions from (which we can call computational paths or simply transaction routes). The users might also constrain this by saying they prefer certain pathways or contracts over others.

A few examples from the past and present

Cowswap

Cowswap utilizes batch auctions as its core price-finding mechanism. Rather than executing trades immediately like AMMs, Cowswap aggregates orders off-chain and settles them in batches. This enables establishing uniform clearing prices across all trades in a batch, eliminating issues like front-running common with immediate execution models. Batch auctions also optimize gas costs by settling many trades simultaneously. An open competition between solvers takes place to submit order settlement solutions that maximize trader welfare for each batch. The best solution sets the finalized uniform prices. Overall, batch auctions bring fairness, efficiency, and MEV protection that immediate execution cannot.

A key innovation enabled by Cowswap’s batch auction model is the ability to find coincidences of wants (CoWs) among orders. CoWs are direct peer-to-peer settlements between trades with reciprocal wants. This liquidity sharing means no external liquidity providers are necessary to facilitate these trades. CoWs can also involve multiple assets simultaneously in ring trades. By maximizing CoWs, batch auctions access more liquidity than isolated pools. Settlements will utilize CoWs when possible, with any remainder executed against on-chain liquidity sources. Combining batch auctions and CoW liquidity sharing gives traders superior pricing and execution.

The CoWswap model is similar to the intent model, where users elicit their intention to trade in the form of a limit order, which goes into the order book, where solvers use the order book state to match them in the form of ring trades or route through AMMs (that is the user only mentions the price, they do not mention the computational path or where exactly they want to execute).

Uniswap-X

The Uniswap X paper proposes a decentralized trading protocol that uses signed off-chain orders and on-chain settlement via Dutch auctions. Users sign orders specifying parameters like input/output tokens, quantities, and price bounds. These orders are disseminated to “fillers” who compete for the best execution price.

Uniswap X suggests setting initial Dutch auction prices through an off-chain RFQ system. Users can poll a network of fillers for quotes and grant a brief exclusivity period to the best quote, incentivizing honest pricing. The order then proceeds to an open Dutch auction.

There are some similarities between Uniswap X and Cowswap:

Both use off-chain signed orders aggregated and settled on-chain in batches. This provides gas savings compared to on-chain orders.
Both aim to find the best execution prices by promoting competition among liquidity providers (Called solvers in the case of cowswap and fillers in the case of uniswapX)
Cowswap emphasizes the coincidence of wants to power direct peer-to-peer trades, whereas Uniswap X focuses more on integrating off-chain and on-chain liquidity sources.

The RFQ (Request for quotation) system in Uniswap X and the signing model with your intent specifying your wishes and letting someone else fill your order are similar to intent architectures.

Towards a formal definition of intents

Instead, users simply express their intent, such as “I want to trade X asset for Y asset.” Sophisticated actors called solvers then figure out how to fulfill that intent optimally, handling all the blockchain specifics behind the scenes. Solvers give proofs that the intent was fulfilled and may participate in mechanisms like auctions to fulfill intents in a decentralized way.

Some definitions are discussed in this blog:

1st Model: An intent $i$ is defined as a tuple $( B , E , T )$:

$B$ represents the set of supported “begin” states.
$E$ represents the set of supported “end” states.
$T$ is a set of preferred sequences of transactions.
The state transition function $s : Q \times T \to Q$ moves from a beginning state to an end state through a sequence of transactions $t$.

An intent is considered fulfilled if it starts in a state $q_0 \in B$ and ends in a state $q_n \in E$ through a transaction sequence $t \in T$.

Intent Clearing: A set of intents $\iota_0, \ldots, \iota_m$ can be cleared if their $B , E , T$ sets have non-empty intersections, allowing the creation of a meta-intent $\iota'$ with the intersections.

As we mentioned earlier, Intents are elicited by the user and then solved by a solver; when it is represented in any such format, the landscape of intents becomes an optimization problem for the solver. A very layman’s way of thinking of this is the user might elicit an intent like “I want to buy 4 ETH worth of BTC,” and the solver would generally find a place to fill or swap this order. But intents don’t stop here; they also allow for constraints to be applied like “Slippage as low as possible”, and “Don’t transact on DEXes that do not allow US users”, which then become additional constraints the solver has to keep in mind.

While there’s another model discussed, the challenges include:

The need for simplified expressions of intents.
Possible welfare implications of specific intents, e.g., zero slippage in DEXs.
Situations where the execution trace may matter due to risks or legal reasons.

The aim is to balance the need for expressivity in capturing user intents and preferences against the practical considerations of computational efficiency and user experience.

There’s also a Lagrangian interpretation of Intent search mentioned here.

To me, the formulation of intents looked like a Markov decision process. Still, while Markov decision processes have stochastic state transitions, this would be a Deterministic MDP with absolute state transition values, which could then be solved with value iteration, policy iteration, or MCTS (Monte Carlo tree search) (This last part was also used to solve the game of Go in Alphago).

Intents as a boon for UX

Intents could be the next stage of UX evolution on-chain. The current way of on-chain UX is the transaction level, where a user signs every transaction that is a part of the action. So, every step on the chain is expressed through a transaction. Intents, in very simple words, are meta transactions where the activity is expressed at a very abstract level, and it is left to the solver to do their best to meet the user’s intent demand. This could include buying some ETH for X amount, and you want the best possible trade, which could either be structured in a single huge swap on Uniswap on Ethereum or sliced across rollups and then bought to Ethereum (calculating fees, too).

Today, a simple swap from USDC to eth would include you approving the limit of the token, approving the token type, and then approving the trade, whereas in an intent-centric world, you could be abstracted away from these details and work with the actions you are interested in taking. An unofficial rule that exists in web design is the idea of no action taking more than three clicks to perform; today, to swap, you have to select both the tokens and perhaps adjust the slippage and the transactions as elicited above, which might not seem like a lot for one swap but over time becomes cumbersome UX.

Unibot, in a very elementary way, offers a glimpse into what intent-specific architecture could look like. It takes away the complex parts of the transaction and offers a simple and accessible UX for traders to trade coins, albeit with some constraints on possible flexibility. Even though there are purported risks around key handling and thus introducing an attack vector, the existence of a consistent user base for the app despite taxes and fees shows unexplored UX opportunities in crypto.

A conversational Intent flow

Where does AI come into the conversation in an intent-centric blockchain world? The idea of intent recognition has been around in Natural language processing for a few decades and is heavily studied in the context of dialogue. Suppose, for example, you go to a travel website and talk to a chatbot; you initially have a purpose, which might be booking the flight or checking on the reservation or status, after which you provide various details. In case of booking a flight, you give the “to” and “from” destinations, the time, the date, and the class of ticket you are interested in; in some instances, you may also have to choose the airport. In this example, your purpose is the intent of the conversation, and the various details you gave are slots that need to be filled to accomplish that intent.

Annotated Dialogue States in a conversation

Another example of intent recognition and slot filling is when you intend to play a song, and there are various slots (details) in the sentence related to the song, like the song name and song artist.

Another Intent Example

In a dialogue world, Intent classification and slot filling is a highly complex problem because your dialogue can span multiple turns, and sometimes there is a global intent and a local intent, and you have to keep track of numerous states. Whenever you’ve used Siri and Google Assistant in the past decade to schedule an alarm or note something in the calendar or birthday, some form of intent classification and slot filling was active in the background.

How does this relate to blockchains? As we move from a transaction-specific world to an intent-centric one, the details of how we go from intents to transactions have yet to be actively present in popular discourse. The interface between the “intent pool” and the “mempool” is non-existent. Accessing On-chain models and using them for intent recognition and slot filling provides one way of a natural language interface intent pools and solvers (and, in my opinion, a most natural one).

The broad idea of this would involve accessing a set of models on a chain that reduces every intent to a DSL (Domain Specific language); this domain-specific language would include details like the core intent (“buy,” “sell,” “bridge,” “borrow/lend”) and have other slots like “address,” “size,” “slippage preferences” (depending on the type of intent). Global DSL allows anyone to deploy a model to reduce intent to this specific DSL. In the presence of multiple such models, a voted majority is taken over the ensemble of models.

The availability of on-chain models helps us develop this interface securely and provably, where proof of computation could be attested for every intent/proof of solving. In some way, capturing the majority voting of various models in some cases might give us insight into how the intent was chosen and, in some rare cases, even help solvers solve these intents better.

The onchain model used here could be a standard deep learning model, like BERT, trained for this very purpose or use inference of large language models in the ensemble; this detail can depend on the various participants or the solvers. In the case of encrypted intent pools, we would need to use Homomorphic encryption or private inference methods to ensure data privacy while still computing over it. Every epoch or every few epochs, a proof could be posted on the chain where the model is the prover. A verifier, either a human or another model, posts a statement about the model’s efficacy. Whether it can still accurately process an intent, this last part ensures that the models’ lifecycle is considered. Sometimes, when the verifier is a sophisticated participant, it might find pitfalls of the model that can be quickly resolved and replaced with a newer model.

As we see below, for the action/idea “buy with stablecoins and dust in my wallet,” once it goes into the intent pool, it is then passed through a variety of models and resolved to a DSL with various details like the intent, the sub-actions and the slots that need to be filled. This resolution to a DSL could be as detailed or abstract as possible; the intent conversation could last a few turns since the threshold of “dust” might have to be determined. Once the DSL is in place, solvers could choose optimal paths to convert these balances to ETH and then pass on the transactions to the mempool.

A sample intent resolution architecture

Another DSL Architecture - Essential

Account abstraction is a concept that decouples accounts from signers in Ethereum by making all accounts smart contracts. This allows accounts to be customized with different authorization logic tailored to users’ needs. However, implementing full account abstraction requires substantial changes to Ethereum’s core protocol.

EIP 4337 takes a different approach to deliver the benefits of account abstraction without consensus layer changes. It introduces “user operations” - pseudo-transactions submitted to an alternate mempool and bundled by “bundlers” into transactions calling an EntryPoint smart contract.

This allows features like social recovery, paying fees in any token, and transaction batching. Developers can build custom accounts adapted to different use cases. By avoiding protocol changes, EIP 4337 can bring these benefits to Ethereum more quickly. However, it does introduce new complexity and actors like bundlers and paymasters. The resulting dynamics around multiple mempools, incentives, and transparency will require thoughtful management.

Intents allow users to specify a desired outcome rather than a specific action. Solvers then help users achieve that outcome in the best way. However, current implementations have limitations around centralization, lack of composability, and inadequate competition among solvers.

An EIP proposed by Essential sets to change this. Account abstraction through initiatives like EIP 4337 enables smart contract-based accounts rather than traditional Externally Owned Accounts (EOAs). This unlocks the possibility for users to submit generalized intents rather than pure transactions. Intents represent a user’s desired outcome and can be combined by “solvers” to maximize participant satisfaction.

EIP 7521 proposes a framework to support an evolving landscape of intent standards without requiring constant smart contract wallet upgrades. Users sign “User intents,” specifying which “IntentStandard” contract should process the intent. These get submitted to an “EntryPoint” contract that handles signature verification as they did in EIP 4337. The User Intents mempool exists alongside the ERC 4337 mempool. Solvers process the intents.

Anoma

Anoma is an intent-centric architecture centered around programmatic intents rather than transactions to build infrastructure layers. Intents are partial state changes signed by users that express preferences, unlike complete state change transactions. This intent-centric design enables decentralized counterparty discovery and solving. The paradigm that Anoma is trying to shift to is a declarative paradigm from an imperative one.

From Adrian Brink’s Talk on Intent Centric apps

Users broadcast intents, which propagate across an intent gossip network. Various nodes can specialize in gossiping about certain intents based on their computational resources and the kind of intents they might want to serve. Solvers observe intents and attempt to combine compatible ones into valid transactions that can be settled on-chain. Transactions are submitted to an encrypted mempool using threshold cryptography, so they cannot be front-run. Anoma also has a partial intent model, allowing for intent combinations.

Chris goes talk on Intent x Rollups - Anoma partial intent model

The focus on privacy when it comes to Anoma is about the “choices at the user level,” i.e., giving a user the flexibility to reveal information about their intents and also choose which parts they want to reveal if they want to do it at all.

The architecture consists of multiple components. The Tiger execution engine handles transparent, shielded, and private data using ZKPs and homomorphic encryption. Typhon is the consensus algorithm. The compiler stack includes the Juvix language, AnomaVM, and VampIR.

The architecture has a homogeneous protocol but a heterogeneous security model. It can be deployed as a standalone blockchain or used to decentralize apps on Ethereum, for instance, as a ZK rollup or to decentralize order books. Users with different security needs can leverage the same protocol while choosing their desired security tradeoffs.

Anoma makes building decentralized applications easier compared to transaction-centric models. Intents enable new applications like runtime rollups, multivariate bartering, and private DAOs. Overall, Anoma provides a flexible and modular architecture tailored to the requirements of contemporary decentralized applications. Focusing on intents rather than transactions solves the issues of counterparty discovery and coordination while preserving privacy.

Anoma ultimately philosophically designed its protocol, looking at intents as “information flows” and “constrained/private information flows” from which the architecture and design choices follow. This also points to the fact that the Anoma intent composition model gives rise to generalized intent models, which may be technically difficult to solve under privacy constraints since efficiency tradeoffs would limit the amount of information that can be private.

Conclusion

Intents currently are a very interesting space within crypto as a research and engineering problem.

Open Problems with Intents:

A formal definition of intent
What do Intent-centric architectures for apps beyond Dexes look like?
The privacy-utility tradeoff to solve any optimization problem is that you need as much information as possible. If private intents are to become a thing, a certain amount of information must be revealed for the intent to be solved.
- What is the minimum knowledge you need to solve an intent problem?
- What are you trading off by cutting access to the rest of the knowledge?
- Formulate this privacy-efficiency tradeoff in a generalized manner.

As spoken about at the end of the last section, A general intent might be too vast to solve for and, in the case of seemingly big state space like ethereum, a very difficult problem to solve. This would point to the fact that Intents are best solved with some constraints and should also be limited when trying to be combined. (when there are common intents). In my opinion, Generalized intents are extremely difficult to implement in practice, and intent-centric architectures will be application-specific in nature.

While these are research problems, various engineering problems also arise with the design choices of implementing intent. It might result in an overreliance (permissioned) middlemen, which risks the centralization of infrastructure across various stacks (In the case of UniswapX, 77% of the volume is filled by off-chain inventory). It can also entrench trusted intermediaries, create high barriers to entry, and stifle innovation, something that is already being seen in MEV. Any intent protocol design must balance permissionlessness, privacy, transparency, and decentralization.

References

Intent-Based Architectures and their Risks — Quintus and Georgios
Anoma: an intent-centric — Christopher Goes
Are Intents, SUAVE, Account Abstraction, & Cross-Chain Bridging all the same thing? — Uma Roy
Realizing Intents with a Resource Model — Chris Goes
Exploring Intents — No Execution podcast
Intent-centric (intent-solver pattern) architectures for fully decentralized dApps — Adrian Brink
The 3rd generation is intent-centric — Adrian Brink
Anoma ❤️ Celestia: intent-centric rollups — Chris Goes
A formalish definition of intents
Lagrangian mechanics of intent solving I — Fabrizio Genovese
Intents with Chris Goes from Anoma — ZK podcast
Wtf is Anoma? Wtf are Intents — Delphi Creative
Anoma
Essential
Cowswap Docs
Uniswap X whitepaper

Designing Multi-Dimensional Blockchain Fee Markets

Mon, 18 Sep 2023 13:28:39 +0000

Upgrading our Fee Markets to the final form.

In this blog, I explore “Dynamic Pricing For Non-fungible Resource: Designing Multidimensional Blockchain Fee Markets.”

buffalu @buffalu__

did you know 58% of solana's compute is spent processing failed arbitrages?

jito-solana contains the systems to efficiently capture this MEV and redistribute it to stakers and validators on the network.

if you want to see a more efficient solana, vote with your stake and stake

Jito @jito

We love Solana's low transaction fees but these have enabled a major issue: MEV spam bots. Fortunately, the Solana community can work together to solve this problem.

A 🧵on MEV and how SOL stakers can increase their yield by fighting spam.

Feb 28, 2023

A natural thing to wonder while viewing this tweet is wondering about the waste of resources and compute. A subsequent question is, what makes it possible? Is it because the resources are being priced too cheap? We’ll come to this problem again in a bit.

Blockchains are public ledgers that allow users to submit transactions that modify the shared state. Full nodes with finite computational resources verify these transactions. How does a transaction try to secure a spot in this block? Many blockchains today enable smart contract development. Smart contracts are basically just programs that run on-chain. Users interact with these programs in the form of transactions, and these are executed by full nodes and added to a “block.” Interacting with these programs in the form of transactions requires some fees the network charges, which can be considered compensation for transaction processing.

What unit of account do these blockchains charge for this? In the case of EVM (Ethereum virtual machine), this unit of account is gas. Each operation in the EVM requires a hardcoded amount of gas. Earlier, we mentioned that blockchains limit the resources consumed in a unit of time. In this case, the unit of time is done on a per-block basis, and the limit is called the block limit, which enforces a “limit” on the total amount of gas consumed. Since there’s only a fixed supply and the demand for this fluctuates, so does the price of “gas” (as demand-supply theory would dictate).

Gas Charge - From Doug Colkitt’s Talk on EVM optimizations

Blockchains face the fundamental challenge of balancing limited computational resources with fluctuating user demands. Transactions compete for block inclusion, consuming network bandwidth, storage, and computation. However, blockchains like Ethereum rely on a fungible unit like gas to price these resources. This makes it hard to reflect each underlying resource’s actual marginal cost and scarcity.

In the paper “Broken Metre: Attacking Resource Metering in EVM,” Perez et al. show a way of a new DoS (Denial of Service) attack on ethereum exploiting this one-dimensional “metering” of resources and propose the Resource Exhaustion attack. The essence of the attack is exploiting the fact that there were EVM instructions for which the gas fees were too low compared to the resources they consumed. This also led to an EIP-2929, which increased the gas requirement for these opcodes for some of the EVM instructions mentioned in the paper. There were also similar attacks like this on the SUICIDE opcode in 2016.

Let’s do a thought experiment: Imagine you live in an apartment building where all utilities - electricity, water, heating, etc. - are bundled into a single bill that all residents split evenly. This means you pay the same monthly amount as your neighbor even if you barely use electricity or water, and they crank up the heat and AC all day. In this scenario, you have no incentive to conserve scarce resources others depend on. You end up overconsuming cheap resources like electricity while underutilizing resources like heat that you don’t personally value as much (or need).

The same issues crop up when blockchains price computation, storage, and bandwidth using one fungible token. You may want to store a large file on-chain while someone else is running complex smart contracts. But you pay gas proportionally to the total resources used, not based on your specific demands. This leads to mispricing and misallocation of scarce non-fungible resources. Computation may be overutilized, while storage is underutilized. There is no way for the market to express the actual marginal cost and demand for each resource.

Ideally, we want the price for each resource to reflect its real-time scarcity so the blockchain charges higher fees when bandwidth is constrained versus when computation is plentiful. This would enable more efficient allocation and usage based on actual supply and demand dynamics.

In their paper “Dynamic Pricing for Non-Fungible Resources,” Diamandis et al. propose a systematic way to define and update dynamic prices for blockchain resources like computation, bandwidth, and storage. This is done by formulating an optimization problem from the network designer’s perspective to maximize transaction utility minus resource loss. The problem decomposes into two parts - minimizing network loss and maximizing transaction producer welfare - joined by resource prices.

Note: This article will be pretty convex optimization heavy, and the reader is suggested to have some familiarity. However, I’ll try to motivate what the equations mean and signify at every step.

Exploration of Fee Markets

Rollups and Data markets

Rollups are a scaling technique that separates transaction execution from data availability. Transactions are executed off-chain in a rollup, with only transaction data posted to the base layer. The roll-up occasionally submits proof of valid state to the base layer. This naturally creates two separate fee markets - one for including transaction data on the base layer and one for execution in the rollup.

Some “lazy” blockchains exclusively are optimized for data availability, leaving the execution to rollups that have also popped up. This also creates separate markets for data inclusion and execution.

Ethereum has proposed allowing notable “blob” transactions containing arbitrary rollup data in EIP-2242 and then expanded upon in EIP-4484. Blobs can be priced separately from base layer gas, creating a two-dimensional fee market. Decoupling execution from data availability unlocks scalability and thus no longer constrains the base chain.

In a proto-danksharding (EIP-4484) implementation, all validators and users still have to directly validate the availability of the full data.

The main feature introduced by proto-danksharding is new transaction type, which we call a blob-carrying transaction. A blob-carrying transaction is like a regular transaction, except it also carries an extra piece of data called a blob. Blobs are extremely large (~125 kB), and can be much cheaper than similar amounts of calldata. However, blob data is not accessible to EVM execution; the EVM can only view a commitment to the blob. - From Proto-Danksharding FAQ

Independent fee markets for data and execution enable more precise price discovery based on actual supply and demand. Overuse of one resource won’t congest the other. Adding a blob/data market alongside base layer gas avoids competition between rollups and base layer transactions. Rollup data has a dedicated channel. Overall, separate fee markets for data availability and execution are a natural fit for rollup architectures. This prevents congestion across purposes and improves efficiency through targeted pricing. The multidimensional fee model proposed in the paper formalizes how to implement such markets.

Proto-dank sharding introduces a two-dimensional EIP-1559 fee market with separate floating gas prices and limits for regular gas and blobs. There are two resources: gas and blobs. Each has a target per block (15M gas, eight blobs), a max per block (30M gas, 16 blobs), and a variable basefee. The blob fee is charged in gas but adjusts based on blob usage to target eight blobs per block on average. Block builders face a more challenging optimization problem balancing gas and blob limits and maximizing revenue. Heuristics can get close to optimal. The exponential EIP-1559 adjustment mechanism for blobs fixes issues with the current EIP-1559 formula, making adjustments depend only on total usage. The fake_exponential function approximates the exponential adjustment while being simple and efficient to compute. In summary, proto-dank sharding creates a two-dimensional fee market to utilize block space better and avoid worst-case usage scenarios. The new exponential blob fee adjustment provides better targeting.

Parallelization

There are two main approaches to enabling parallel execution in blockchains:

Minimal VM changes and the responsibility is shifted to full nodes to identify parallelization opportunities.
Using access lists - Transactions specify which accounts they access so that Non-conflicting transactions can be executed in parallel. This is used in Solana Sealevel to unlock parallel processing for thousands of transactions.

The issue with the access list approach is that contention for popular accounts limits parallelization gains. Many transactions want to access the same accounts (NFT mints, popular protocols, or presales), forcing sequential execution.

We also saw Easy Parallelizability being discussed for Ethereum, and this was discussed as a way to speed up the EVM in a blog by flashbots. In the end, we will see a formalization of fee markets for threaded VM resource pricing.

Exploration by Ethereum and Solana

In ethereum, a multidimensional fee market was proposed by Vitalik in the form of Multidimensional EIP1559.

Ethereum has resources with different bursts (short-term) and sustained (long-term) capacity limits. For example, the EVM can handle occasional slow blocks but not sustained ones. The current gas model doesn’t handle these burst vs. sustained differences well. It makes worst-case and average-case ratios similar, leading to inefficient gas costs, but the resources used in these cases are vastly different. For example, on average, transaction data plus call data consumes ~3% of the gas in a block. Hence, a worst-case block contains ~67x (including the 2x slack from EIP 1559) more data than an average-case block. The post proposes a multidimensional EIP 1559 - separate EIP 1559 controllers for each resource. Base fees for each resource are adjusted separately based on usage. This allows much higher slack parameters as the entire burst/sustained gap is represented. Limits would rarely be hit except in edge cases. Resources could include EVM execution, call data, witness data, and storage. The benefits noted from this were lower fees from more efficient pricing, better DoS protection, and reduced need for dynamic basefee algorithms. We will not cover a thorough analysis of EIP 1559 and transaction fee mechanism in this blog and refer the reader to Transaction Fee Mechanism Design for the Ethereum Blockchain: An Economic Analysis of EIP-1559 and Transaction Fee Mechanism Design by Tim Roughgarden et al. (We will cover these works in a later blog). One thing to note about EIP-1559 is that although it is close to the problem that this paper considers, it makes the fee estimation problem easier in a way that disincentivizes manipulation and collusion. This paper aims to price resources dynamically to achieve set objectives.

Multidimensional Fee markets are also being considered in a different form. Anatoly Yakovenko proposed it for Solana in “Consider increasing fees for writable accounts.” The issue proposes increasing fees exponentially for unused writable accounts to disincentivize bots flooding the network with invalid transactions based on stale state. The fee design is meant to help build congestion control, capture fees, punish misbehaving senders, and refund well-behaved senders. An alternate proposal suggested increasing fees for used accounts and rebating a portion of the program, giving developers more control while still limiting contention.

Defining the Problem

Transactions A transaction in ethereum is defined as any action taken by an externally owned account (i.e., not a smart contract). Suppose A sends B 1 eth. This must be debited from A’s account and credited into B. A transaction changes the state of the EVM.

The data (arbitrary in nature) is sent over the p2p (peer-to-peer) network to be added to the chain. They are first broadcasted and collected by nodes in the mempool. The mempools act as a staging/holding area for pending transactions waiting to be included in the block. Transactions in the mempool are prioritized by various factors, primarily the gas price offered. This is why gas immediately goes up during high volume transaction times because users keep spamming transactions with higher gas fees, so their transaction gets included and is prioritized. A miner/validator gets to choose which transactions from the mempool go into a block. Miners may also outsource this to “block builders.”

Nodes Nodes execute and validate transactions to maintain the latest state. Most blockchains try to have minimum computational requirements for these nodes in a blockchain. They have finite resources, so blockchains limit total resources per unit of time to prevent overload. If transactions are included in a blockchain faster than nodes can execute them, these nodes won’t be able to reconstruct the latest state and, therefore, assert validity. This is also called Resource Exhaustion Attack.

Resource Targets and Limits One way to avoid Resource Exhaustion would be to enforce a fixed upper bound of resources/combination of resources in a unit amount of time (or blocks). Another way would be to ensure miners do not constantly include high-resource transactions to blocks and better manage resource distribution/loading. This would suggest that we should have a “Resource Target” - a minimum target for consumption of resources every block to avoid waste of resources and a “Resource Limit” - so we make sure nodes catch up after a while.

Resources

Most blockchains have several “meterable” resources (as in measurable, and we use this to quantify their usage as well). We will label these resources $i = 1, \ldots, m$.

In Ethereum’s EVM, the fee unit is gas. Each operation consumes a hardcoded gas amount. As seen below, they all consume resources. We will henceforth only consider resource consumption rather than these granular opcodes (although you could use them if you wanted to).

Opcodes and their Gas Cost

For a given transaction $j = 1, \ldots, n$, we will let $a_j \in \mathbb{R}^m_+$ be the vector of resources that transaction $j$ consumes. In particular, the $i$th entry of this vector, $(a_j)_i$, denotes the amount of resource $i$ that transaction $j$ uses. Basically

If we have three resources that we are trying to measure, let’s say storage, computation, and bandwidth. For some transaction $x$ we could write the vector as $[\text{storage}, \text{computation}, \text{bandwidth}]$. In more concrete terms, taking numbers here $[10, 20,30]$ would be the different resources transaction $x$ consumes. These quantities don’t need to be non-negative (E.g., some resources may not consume storage, so that they would be 0).

Okay, another question to ask is, can we consider a combination of resources? What if computation alone doesn’t have much meaning to be considered, and you want to consider computation and bandwidth both? Or what if using computation or storage alone is okay, but using them both is costly? In this case, formally, if you have two resources $R_1$ and $R_2$ you can create a combined resource $R_1R_2$ which can be metered separately. So your vector could look like

$$x = [ R_1, R_2, R_1R_2]$$

Now that we have representation for resources, we will talk about resources we want to utilize. As we spoke earlier, we want a “resource target,” i.e., we would like to have a sustained amount of resources always being used in the blockchain (ideally). In the case of ethereum, there is a resource target for gas, which is 15M gas per block. Now, since we want to expand it to multiple resources, we will represent it by a vector $b^* \in \mathbb{R}$ with the exact representation as in the resource case, the ith entry denotes the desired target of resource i in a block. The resource utilization of a particular block is a linear function of transactions included in a block written as a Boolean vector $x \in \{0,1\}^n$, which is again a vector.

To understand this as a simple example, let’s say we have five transactions being considered, and the 1st, 2nd, 4th, and 5th are being included in the block. The boolean vector looks like

$$x = [1, 1, 0, 1, 1]^T$$

Since we have multiple transactions and each transaction consumes its resources, we will represent this as a matrix $A \in \mathbb{R}^{m \times n}$ whose $j$th column is the vector of resources $a_j$ consumed by the transaction $j$. What do we mean? Continuing our example, we have five transactions and three resources each (as mentioned earlier, computation, storage, and bandwidth). So, an example matrix looks like

$$A = \begin{bmatrix}12 & 32 & 34 & 43 & 54 \\1 & 2 & 3 & 4 & 5\\10 & 20 & 30 & 40 & 50\end{bmatrix}$$

If we want to write this block’s total quantity of consumed resources, we will write it as $y = Ax$.

In our example

$$y = Ax$$$$y = \begin{bmatrix}12 & 32 & 34 & 43 & 54 \\ 1 & 2 & 3 & 4 & 5 \\ 10 & 20 & 30 & 40 & 50\end{bmatrix} * \begin{bmatrix}1 \\ 1 \\ 0 \\ 1 \\ 1 \end{bmatrix}$$$$y = [141, 12, 120]$$

where the $i$th entry is the $i$th resource consumed by all transactions included in the block (for reference $[\text{storage}, \text{computation}, \text{bandwidth}]$).

Now, if we want to measure the deviation of resources from our target, we would subtract our total resources used and the resource target we had. So, we want to measure the deviation between the Resource total given by $Ax$ and the Resource Target $b^*$ Formally

$Ax-b^*$,

is another vector whose $i$th element gives the deviation for the $i$th resource. (positive or negative, positive would mean more resources used, negative would mean less resources used compared to resource target).

Finally, we also mentioned we will have a Resource limit. We don’t want any valid block to use resources greater than the resource limit we set. We will represent it by $b$.

Formally, $Ax \leq b$. In the case of ethereum, gas (single resource) has a limit of 30 million gas per block.

There we have it. We have represented how to measure the various resources, calculate deviation, and represent the resource limit.

Network Fees

Now that we have formulated the things we want. We will now express the fees. As we mentioned earlier, fees are charged by the network for transactions (i.e., for operations that are done in a transaction). We want to ensure its set so that the usage is close to the resource target, not the limit. If transaction $j$ with resource vector $a_j$ is included in a block, a fee $p^Ta_j = \sum_ip_i(a_j)_i$ is paid to the network. A natural thing to assume now is that as the resource keeps getting scarce, any additional amount required is costlier than it would have been otherwise (supply-demand law).

Note: In Ethereum, the network fee is implemented by burning some amount of gas (post EIP 1559).

Resource Mispricing Given $A$ (the block of transactions) and $b$ (the resource limit), it is not very obvious how to set fees $p$ to ensure the network performs well.

One Example we should note is the transaction spam attack, which is called the EXTCODESIZE opcode repeatedly, an attack that happened in 2016. This exploited mispriced resources since EXTCODESIZE was too cheap and caused the network to slow down a lot. When you attempt a full node today, you can see a dramatic slowdown and processing from block 2283416 to 2463000. This immediately led to a subsequent EIP-150, which fixed the issue with the hardfork. This incident makes us realize the importance of resource pricing appropriately and not neglecting this problem since it can directly affect the performance of the underlying chain.

A simple Idea

Lets now think of a simple rule for $p$:

if $Ax =b^*$, there is no update since we have reached the resource target.
If $(Ax)_i >g b^*_i$, increase $p_i$ to reflect this.
If $(Ax)_i < g b^*_i$, decrease $p_i$ since we have not reached the basic resource target.

How do you update $p$ in the next block, though? This equation can be given by

$$p^{k+1} = \left(p^k + \eta(Ax - b^*)\right)_+$$

where,

$p^k$ is the vector of prices for each resource at iteration $k$
$\eta$ is the learning rate - this controls how big of an update step we take. If we make this too big, then the jump in the fees from one block to the next will be too high, thus making the network unusable. If we make it too small, the fees might be too low to reflect the demand that the resources are under right now and, hence, might lead to spam.
$Ax$ is the observed resource utilization from the transactions included in the latest block
$b^*$ is the target resource utilization

In words, it is:

New prices = Current prices - Learning rate * (Observed usage - Target usage)

(A slight aside for interested readers: This equation is an example of the gradient descent update rule for iteratively adjusting the weights of the network, but instead of $Ax-b^*$, you would have the loss.)

This type of iterative price update rule based on supply and demand motions (a different motivation in other spaces) appears in many contexts. Still, it is especially prevalent in machine learning for optimization and multi-agent learning. The goal is always to gradually steer behavior toward an optimal point, just like tuning the prices in a blockchain fee market.

Resource Allocation Problem

The ultimate goal of this system is to maximize the utility of the underlying blockchain. Since we are not omniscient, we do not know what the inclusion of a transaction in a block means to a user or miner (i.e., the utility) or what they want to add to a block. So, most of our optimization/design is around the desire to modify the fees so that resource usage is always near the resource target.

Loss Function - We define a loss function to somehow measure the “dissatisfaction” or the unhappiness of the network designer. We assume only that $\ell$ is convex and lower semicontinuous. Assuming a function is convex makes it more efficient to optimize and also makes it so that the function has one optimal (global minima in the case of a convex function) value (that we are trying to find). Semicontinuous means that the optimization process is stable and that some minimum exists.

We observe two equations below. The first design of the loss function says we are only okay if the resource utilization $Ax$ or, in other words, $y$ is always equal to the resource target $b^*$. The second one basically encodes the notion that we are happy if $y$ is less than $b^*$ but unhappy otherwise.

$$\ell(y) = \begin{cases} 0 & y = b^* \\ \infty & \text{otherwise.} \end{cases}$$$$\ell(y) = \begin{cases} 0 & y \leq b^* \\ \infty & \text{otherwise.} \end{cases}$$

While there are other losses we can consider, one important one is calculating the per-resource utilization.

$$\ell(y) = \sum_{i=1}^{m} \phi_i(y_i)$$

Each loss function in some form ultimately expresses what the network designer wants to optimize for and capture tradeoffs as well. These choices result in a separate update rule (like the one we saw earlier) for the network fees $p$.

Resource Constraints $S$ represents the set of valid transaction bundles users and miners can create and include in a block. This set can encode various constraints:

Hard limits on resource consumption like $Ax \leq b$
Contention for popular accounts or contracts
Interdependencies between transactions

So, $S$ captures all the limitations and complexities around which transactions can be bundled together in a block. A reader who has followed us till now might be confused about why $S$ exists in the same world as $x$ since we mentioned that $x$ also denotes the transactions included in the block. The best way to think about this is $S$ defines the finite, discrete set of transaction bundles that could end up in $x$ based on real-world constraints. $S$ represents “This is what is actually possible,” while $x$ represents “This is what miners chose.” This is also why we have the constraint $x \in S$.

Convex hull of Resource Constraints The convex hull $conv(S)$ contains all convex combinations of points in $S$. This allows the network designer to “average” transactions over multiple blocks. Specifically, components of $x$ can vary continuously between 0 and 1, interpreted as the probability or fraction of including a transaction over many blocks. Taking the convex hull $conv(S)$ relaxes the binary constraints, allowing “partial” transactions. This relaxation allows the designer to reason about long-term resource utilization rather than allocation in a single block. This does not mean users or miners will create partial transactions. It’s a convenience for the designer. Users/miners can only include transactions fully or not at all. The convex hull allows the designer to model allocation in an idealized way. This will allow the problem to decompose nicely into two coupled subproblems: one solved on-chain and one off-chain with integral solutions.

Transaction utility We define transaction utilities in the form of $q \in \mathbb{R}^n$. $q_j$ represents the joint “utility of miners and users” for including transaction $j$ in a block of $n$ transactions. This is an opaque quantity and very hard to know. The actual values of $q$ don’t need to be known. Only users/miners try to maximize utility.

The problem

$$\begin{aligned} \text{maximize} \quad & q^Tx - \ell(y) \\ \text{subject to} \quad & y = Ax \\ & x \in \mathbf{conv}(S). \end{aligned}$$

Where,

$q^Tx - \ell(y)$ represents maximizing total transaction utility
The constraint $y = Ax$ enforces that the resource utilization $y$ must equal the resources consumed by the chosen transactions $Ax$.
The constraint $x \in conv(S)$ requires the transaction bundle $x$ to be an element of the convex hull of the feasible set $S$. This allows “partial” transactions.

The interpretation is that this is the “ideal but unrealistic” problem the network designer would solve. The paper shows that this decomposes into two problems: on-chain and implicitly by users and miners. The combination is called transaction producers. This combination exists because it’s inevitable that a user-miner colludes. This possibility was also mentioned in the paper “Transaction Fee Mechanism Design” by Roughgarden et al.

The Dual function

Okay, we defined a problem we couldn’t solve and now need to convert it into a form, so we will.

When dealing with a complex optimization problem, the concept of duality is a powerful tool. The dual problem is a natural counterpart to your original, or “primal,” optimization problem. The dual problem is always the opposite of the primal. If the primal is minimization, the dual problem is maximization and vice versa. The dual is appealing because, more often than not, original problems in convex optimization seem to be very difficult but become very solvable in the dual form. What also makes the dual so compelling is that it provides a lower bound for the solution of the primal problem. This is invaluable because it gives you a benchmark for how good your solutions can be. They offer you alternative perspectives for understanding your problem, simplify complex equations, and can even lead to more efficient algorithms.

In the primal problem above, we were optimizing a constrained optimization problem.

We first pull in the $x \in conv(S)$ into our resource constraint problem to obtain

$$\begin{aligned} \text{maximize} \quad & q^Tx - \ell(y) - I(x) \\ \text{subject to} \quad & y = Ax \end{aligned}$$

where $I$ is an indicator function which is 0 if $x \in conv(S)$ and $+\infty$ otherwise. (As we said earlier, we want our $x$ only to be within what’s possible. It’s meaningless to us otherwise).

We add the constraints to the objective function to form the Lagrangian using dual variables (Lagrange multipliers). The dual for this problem is defined by

$$L(x, y, p) = q^Tx - \ell(y) - I(x) + p^T(y - Ax),$$

where the constraint gets added to the original problem and $p^T(y-Ax)$ represents a penalty.

Rearranging, we finally find the dual function,

$$g(p) = \sup_y \left(p^Ty - \ell(y)\right) + \sup_x \left((q - A^Tp)^Tx - I(x)\right).$$

Where,

$p$ is the Lagrangian dual variable (also called the price) that relaxes the equality constraint $y = Ax$

We aim to maximize the Lagrangian $L(x, y, p)$ over $x$ and $y$.

The first time in the above equation is the Fenchel conjugate of the resource constraint problem evaluated at $p$. We will write it as $\ell^*(p)$.

The Fenchel conjugate is a key concept in convex analysis and duality theory. Given a function $f(x)$, the Fenchel conjugate $f^*(y)$ is defined as:

$$f^*(y) = \sup_x \left( y^T x - f(x) \right)$$

Where “sup” refers to the supremum or least upper bound.

Intuitively, $f^*(y)$ represents the maximum value that can be obtained by matching the function $f(x)$ with a linear function $y^T x$. It captures the best possible alignment between $f(x)$ and its linear approximation.

Some key properties of the Fenchel conjugate:

$f^*(y)$ is a convex function, even if $f(x)$ is nonconvex. This makes it very useful in convex optimization.
The conjugate “flips” maximization and minimization. Maximizing $f^*(y)$ is equivalent to minimizing $f(x)$.

The second term defines the transaction producers’ problem, and it optimizes the following problem

$$\begin{aligned} \text{maximize} \quad & (q - A^Tp)^Tx \\ \text{subject to} \quad & x \in \mathbf{conv}(S), \end{aligned}$$

Where,

$q$ is the transaction utility vector. $q_j$ is the utility of including transaction $j$ in the block.
$A$ is the resource matrix, where column $j$ is the resource vector $a_j$ consumed by transaction $j$.
$p$ is the resource price vector set by the network.
$x$ is a binary vector indicating which transactions are included in the block.

So $(q - A^Tp)^Tx$ is the total utility to transaction producers of the transactions included in $x$ (the utility $q_j$ of each transaction $j$, minus the fee paid to the network $p^Ta_j$ for that transaction’s resource usage).

This is maximized subject to the constraint $x \in conv(S)$, which means $x$ is in the convex hull of the set of possible/valid transaction bundles $S$.

So, in other words, the second term is finding the set of transactions $x$ that maximizes the transaction producers’ total utility, subject to $x$ being a valid transaction bundle. We will write the second term regarding $p$ as $f(p)$. This is a convex function.

We write the dual function as

$$g(p) = \underbrace{\ell^*(p)}_{\text{network}} + \underbrace{f(p)}_{\text{tx producers}}.$$

Since both the parts are convex, $g(p)$ is also a convex function.

The dual problem

$$\text{minimize} \quad g(p),$$

The dual problem is to minimize $g$ as a function of the fees $p$. By optimizing over $p$, We align incentives across the network and transaction producers to achieve an optimal system-wide outcome.

Under certain regularity conditions, the functions $\ell^*$ and $f$ are differentiable, and their gradients can be characterized as:

$$\nabla \ell^*(p) = y^*$$$$\nabla f(p) = -Ax^*$$

where $y^*$ maximizes $p^Ty - \ell(y)$ and $x$ maximizes $(q - A^Tp)^Tx$ over $conv(S)$.

Therefore, the gradient of the overall dual function is:

$$\nabla g(p^*) = y^* - Ax^* = 0.$$

It states that the optimal prices $p^*$ equalize the resource usage target $y^*$ with the realized usage $Ax^*$ induced by the transaction producers at those prices.

In essence, the optimal fees align the incentives of the network (to minimize its loss $\ell(y)$) and the transaction producers (to maximize their utility) by properly internalizing the costs. The optimal fee that should be charged is the exact marginal cost the network faces.

Now, we provide conditions under which the optimal resource prices/fees $p^*$ will be non-zero.

We define two disjoint sets, where $X^*$ is the optimal set of transactions included when resource fees are 0 and $Y^*$ is the set that minimizes the loss.

$$X^* = \operatorname*{argmax}_{x \in \mathbf{conv}(S)} \; q^Tx,$$$$Y^* = \operatorname*{argmin}_{y} \; \ell(y),$$

The condition we then have is

$$AX^* \cap Y^* = \emptyset,$$

In other words, if the resource usage $AX^*$ induced by the zero-fee demand $X^*$ does not overlap at all with the optimal usage $Y^*$ that minimizes network loss, then the optimal prices $p^*$ must be non-zero. Intuitively, this means that if the users/miners want to include transactions at zero price that incur some network loss, then the network must charge non-zero fees $p^*$ to align incentives. Fees guide behavior away from zero-fee demand $AX^*$ toward the optimal usage $Y^*$.

This motivates the need for multidimensional pricing. With separate prices for each resource, over-utilization of one resource can be discouraged by increasing its fee, while under-utilization of another can be encouraged by decreasing its fee.

There are some properties we can derive from dual problem about the prices $p$:

First, the optimal price $p^*$ must be non-negative.
Second, it defines a condition on the loss function called superlinearity, which implies that the domain where the dual function $g(p)$ is finite is precisely the nonnegative orthant. This means the optimal prices are restricted to be non-negative. Superlinear losses prevent unbounded subsidies.
Third, it characterizes the maximum prices beyond which transactions that consume resources will not be included by users/miners. This helps bound the range of reasonable fee values.

Overall, the properties guide setting resource prices and characterizing their behavior. The nonnegativity constraints reflect the increasing costs of higher network usage. And the maximum prices bound subsidies and prevent exclusion of all resource-using transactions. The takeaway is that despite its generality, the structure of the network loss function $\ell(y)$ and dual function $g(p)$ enable valuable insights into the optimal pricing of blockchain resources.

The Solution

We can iteratively converge to the optimal prices. In a less constrained environment, L-BGFS could work.

But since on-chain environments are compute-constrained (like that on ethereum), the paper suggests a modified version of gradient descent that is easy to compute and does not require storage beyond the fees.

In gradient descent, we have

$$p^{k+1} = p^k - \eta \nabla g(p^k).$$

Where $\eta$ is the learning rate and if $g$ is differentiable, we use the derivate to update the prices. This works well when $g(p)$ is differentiable. The update direction is guaranteed to reduce $g(p)$.

To ensure $p$ stays in the domain of $g$, the paper suggests using projected gradient descent:

where $\mathbf{proj}(z)$ projects $z$ onto the domain of $g$.

Now, evaluating the function $g$ at point $p^k$ is not always possible. We saw earlier if $g$ is differentiable, it only depends on the solutions to the two terms.

Let $y^*$ be a maximizer of the $\sup_y(p_k^T y - \ell(y))$, which is easy to compute in practice, and we replace $x^*$ with the observed solution $x^0$. Since this $x$ is boolean, we compute resource usage $Ax^0$ after observing the included transactions.

We can then use an updated form.

$$p^{k+1} = \mathbf{proj}(p^k - \eta(y^* - Ax^0)).$$

We see that this equation increases the network fee for a resource being overutilized and decreases the network fee for a resource being underutilized. As you would expect, this pricing mechanism is designed to disincentivize future users and miners from including transactions that consume currently overutilized resources in future blocks. This is not the only algorithm that works for this.

The paper gives a few examples of loss functions and the update rules we get from them.

Consider our earliest loss function,

$$\ell(y) = \begin{cases} 0 & y = b^* \\ +\infty & \text{otherwise,} \end{cases}$$

The conjugate function we get is

$$\ell^*(p) = \sup_y(y^Tp - \ell(y)) = (b^*)^Tp,$$

where the optimal value of $y$ is $b^*$. The update rule is,

$$p^{k+1} = p^k - \eta(b^* - Ax^0).$$

Consider another loss function: linearly separable losses

$$\ell(y) = \sum_{i=1}^{m} \phi_i(y_i).$$

The loss comes out to be

$$p^{k+1} = (p^k - \eta(b^* - Ax^0))_+.$$

While the papers use the gradient descent rule for this, other modifications can be considered, like adding momentum and adaptive steps.

Extensions of the Fee Market

Parallel Transaction Execution

Consider $L$ parallel execution threads, each with its resources plus shared resources. The transaction run on thread $k$ is denoted by $x_k \in \{0,1\}^n$.

The allocation problem is given by:

$$\begin{aligned} \text{maximize} \quad & \sum_{k=1}^{L} q^Tx_k - \ell(y_1, \ldots, y_L, y^{\text{shared}}) \\ \text{subject to} \quad & y_k = Ax_k, \qquad k = 1, \ldots, L \\ & z = \sum_{k=1}^{L} x_k \\ & y^{\text{shared}} = Bz \\ & z \in \mathbf{conv}(S^{\text{shared}}) \\ & x_k \in \mathbf{conv}(S), \qquad k = 1, \ldots, L. \end{aligned}$$

where,

Transactions $x_k$ are allocated to thread $k$.
Resource usage is $y_k = Ax_k$ per thread, $y^{\text{shared}} = Bz$ for shared resources, where $z = \sum_k x_k$.
Resource allocation problem is to maximize utility minus loss.
This can be solved using the same duality approach by combining all resources into one vector.
It enables pricing threads and shared resources separately.

Different Price Update Speeds

The premise is that some resources can sustain burst capacities for much shorter periods of time compared to other resources. Therefore, Some resources may need faster price adjustments than others.
Introduce per-resource learning rates $\eta_i$ to update resource prices $p_i$ for resource $i$ faster when needed.
Update rule becomes

$$p^{k+1} = \mathbf{proj}(p^k - D\nabla g(p))$$

where, $D = \mathrm{diag}(\eta_1, \ldots, \eta_m)$.

We can also define this problem on a per-contract utilization basis instead of per resource.

Conclusion

We explored multi-dimensional fee markets in this blog. In future blogs, I will explore transaction fee mechanism design.

Understanding "Towards a Theory of MEV II"

Thu, 14 Sep 2023 17:21:18 +0000

Exploring the second part of the Theory of MEV. :D

Blockchain systems rely on decentralized networks of participants to maintain a global shared state and execute transactions (in the case of ethereum, validators). A core challenge is dealing with varying incentives and strategic behavior by different network participants. In particular, validators who produce blocks have temporary power to reorder or include transactions, enabling profit at the expense of others. This phenomenon of excessive profits extractable by temporary inclusion monopolists by reordering or inserting transactions is known as Maximal Extractable Value (MEV).

In an earlier work, “Towards a Theory of MEV I,” Kulkarni et al. study the game theoretic properties of MEV, explicitly answering questions about how reordering impacts the price of sandwich attacks and how sandwich attacks affect routing MEV. They find that of reordering MEV, the maximum price impact caused by reordering of sandwich attacks on the order of $O(\log n)$ where $n$ is the number of user trades and that in the case of routing MEV, the existence of MEV degrades and counterintuitively improves the routing quality. For a quick recap, you can look at a talk by Tarun Chitra and my blog.

In this blog, I cover the work “Towards a Theory of MEV II: Uncertainty.” While the first paper focused on analyzing MEV for constant function market makers specifically, this work aims to develop a more general theory around payoffs (more on what payoffs are later).

From Tarun Chitra’s talk at Modular Summit

Ultimately, blockchain users encounter a variety of uncertainty; as the image shows, it’s on a spectrum. This article only covers the first part and is about Ordering Uncertainty. Ultimately, MEV can be viewed as sophisticated participants capturing profits from user uncertainty, and one more way of looking at it is capturing profits from user inefficiencies.

This paper provides a quantitative trade-off between the freedom to reorder transactions and the payoffs in decentralized networks. The results can be viewed as an analog of sampling theorems, showing simple rules suffice for simple payoffs, but more complexity is necessary to limit unfairness for arbitrary payoffs. The technical core involves Fourier analysis of the symmetric group, representation theory, and uncertainty principles. These characterize the relationship between sequencing rules, payoff complexity, and fairness guarantees.

MEV Mitigation

Fair Ordering has been introduced as a solution to handle MEV mitigation. These methods propose changing blockchain consensus, forcing validators to adhere to a particular order of transactions. This order can be determined by time or some other rules. One common form of Transaction ordering is FCFS (first come, first serve), where validators note the times of the transactions they receive, and the one that comes earlier appears early in the order as per the fair ordering protocol. In two recent papers, “Order-Fair Consensus in the Permissionless Setting,” Kelkar et al. and “Themis: Fast, Strong Order-Fairness in Byzantine Consensus,” Kelkar et al. propose ways to achieve such fair Ordering.

However, research in social choice theory renders such attempts moot. How? Social choice theory is a field of economics and political science that deals with creating frameworks that explore how individual preferences can be aggregated to make collective decisions. Two important results in Social choice theory make it hard to design universal fair ordering protocols.

The Condorcet paradox demonstrates that majority preferences can be inconsistent under certain conditions because of the intransitive preferences of choices among voters. For example:

Suppose there are three voters - Alice, Bob, and Charlie. They are trying to decide among three options - A, B, and C. Their preferences are:

Alice: A > B > C
Bob: B > C > A
Charlie: C > A > B

If we tally the votes between each pair of options:

A vs. B: Alice and Charlie prefer A over B, so A wins 2-1
A vs. C: Bob and Charlie prefer C over A, so C wins 2-1
B vs. C: Alice and Bob prefer B over C, so B wins 2-1

So the majority prefers A > B; B > C; C > A

This creates a paradox or inconsistency since transitive logic would require that A > B, B > C, and C > A is impossible.

The Arrow impossibility theorem builds on this idea and makes an even stronger statement about the difficulty of aggregating individual preferences into coherent group choices. In essence, it proves that aggregating individual preferences into a rational collective preference ordering is impossible when there are three or more options and two or more people. This implies no perfect voting system; all democratic voting methods involve trade-offs between fairness criteria. For more on Arrow’s theorem, read this.

Does this mean giving up on any transaction ordering is the only solution? In “Credible Decentralized Exchange Design via Verifiable Sequencing Rules,” Ferreira et al. show that if one aims to design a sequencing rule where the miner can never obtain risk-free profits (risk-free profits meaning the miner is sure to receive some tokens for free), such a goal is unattainable (As far as deterministic sequencing rules are concerned). They also show that some sequencing rules increase social welfare for unsophisticated agents (normal protocol users).

Social welfare functions are mathematical models used to aggregate individual preferences into a single measure that represents the “well-being” of a group. This paper’s authors believe that Ordering’s impact on AMMs is only possible partly due to the convexity of the payoff function of an AMM. If social welfare functions can restrict orderings to mitigate the adverse welfare effects of MEV, a natural question is, what is a generalized way of constructing such orderings? The authors try to generalize these rules to a larger class of payoff functions.

Uncertainty Principles

Formal Definition: The Heisenberg Uncertainty Principle in quantum mechanics is usually stated as follows:

$$\Delta x \Delta p \geq \hbar/2$$

Here, $\Delta x$ is the uncertainty in position, $\Delta p$ is the uncertainty in momentum, and $\hbar$ is the reduced Planck constant, approximately $1.054571 \times 10^{-34}$ $\mathrm{m^2 kg/s}$.

In Simpler Words, Imagine you’re trying to look at a tiny particle, like an electron, under a super-microscope. The Heisenberg Uncertainty Principle says that you can’t know both where the particle is (its position) and how fast and in what direction it’s moving (its momentum) with perfect accuracy at the same time. It’s like trying to watch a fast-moving hummingbird; if you focus on capturing its speed, you can’t pinpoint its exact location, and vice versa.

The uncertainty principle generally states that there are inherent limitations in our ability to measure or know pairs of complementary properties/functions at the same time. The general mathematical form often takes the shape of an inequality, like

$$C(f) \cdot C(Lf) \geq c$$

where $C(f)$ and $C(Lf)$ are measures of “complexity” or “uncertainty” for a function $f$ and its transformed version $Lf$, $c$ is a constant greater than zero, and $L$ is the linear transform. This inequality tells us that if one of the measures is small (i.e. if we know one property very precisely), the other must be large enough to satisfy the inequality, meaning the other property cannot be known with arbitrary precision.

In the case of the Heisenberg uncertainty principle, one of the attributes is indeed related to the Fourier transform of the other. Specifically, the position $x$ and momentum $p$ are Fourier transform pairs. In quantum mechanics, the wave function $\psi(x)$ in position space can be transformed into momentum space $\phi(p)$ using the Fourier transform.

This paper aims to construct complexity measures $C$ representing the fairness of payoff functions $f$. The complexity measures will connect the sizes of particular subsets of permutations (basically transaction orderings) to $C(f)$. These subsets are outputs of sequencing rules (hence, they also allow us to talk about different “kinds” of orderings in general).

Background

Blockchains

Blockchains are decentralized systems that maintain a sequence of blocks containing state transitions and come to a consensus on a particular state. Each block consists of transactions that mutate the state from the previous block. Users submit transactions via a peer-to-peer network and pay fees to include them in a block. Validators or miners collect transactions, validate blocks, and add these to the global ledger. Validators lock up resources to participate in the overall consensus protocol and are incentivized by fees and block rewards. A key aspect of blockchains is that they allow asynchronous communication about a shared (global) state, with protocols designed to make it costly for validators to deviate from consensus. While adding blocks to the blockchain, a temporary monopoly is given to a single validator in many consensus protocols.

MEV

This temporary monopoly of allowing validators to reorder or censor transactions and earn fees and profit from this practice is called Miner extractable value.

This paper assumes that there is a fixed set of transactions $T_1, \ldots, T_n$ and a payoff function $f(T_1, \ldots, T_n)$ that yields profit to the validator. This paper does not consider the difference between the allocation/split of the profit between the validator and searchers (entities that search for profitable orderings and submit them to the validator).

Payoff Functions

The payoff function $f$ is a mapping from transactions to a payoff. Since we consider a fixed set of transactions for reordering MEV, the payoff to validators from executing transactions in a particular order can be modeled as a function $f$ that maps permutations of the transaction set to real numbers.

Some notation

$S_n$ → set of permutations of $n$ elements
$\pi$ → permutation; $\pi(i)$ → index where ith element is moved

The function $f: S_n \to \mathbb{R}$ is a payoff function that takes a transaction ordering and gives a payoff. Concretely, for a function $g: T^n \to \mathbb{R}$, we have

$$f(\pi) = g(T_{\pi(1)}, \ldots, T_{\pi(n)})$$

The Ordering is the optimal monopoly profit achievable via reordering MEV. Here, the max denotes the permutation out of the set of permutations of $n$ transactions that gives us the maximum payoff.

$$\max_{\pi \in S_n} f(\pi)$$

Constant Function Market Makers: Let’s think about the trades in an AMM briefly. Users submit transactions and are then added to a block. How is MEV extracted around these? Sandwich attacks are the most common form of MEV extraction. In a sandwich trade, an order is placed before and after the submitted transaction by the user, and the slight price difference (impact) is booked as the profit. The order in which user trades are executed changes the profitability. So, the payoff function $f(\pi)$ depends on the permutation $\pi$ of user trades.

The payoff function for CFMMs is given by:

$$f(\pi) = \sum_{i=1}^{n} \mathrm{PNL}(\Delta_{\pi(i)}, p_{\pi(i-1)})$$

$\Delta_i$ denotes the transaction by the user, and $p_i$ is the price obtained after the execution of the user transaction. This payoff is not permutation invariant. Some permutations are clearly better than others.

Liquidations: Liquidations allow protocols to close positions that have become undercollateralized due to price changes. For example, a user borrows asset $Y$ by posting asset $X$ as collateral at an initial price ratio of $p_0$. If the price of $X$ in terms of $Y$ later drops to $p^\star$ (the liquidation price), liquidators can profitably close the position.

Liquidation is generally very permutation sensitive, i.e., for some orderings (maybe even most), there might not be a liquidation at all. For a permutation $\pi$ of trades, define $p_i(\pi)$ as the price after trades $\pi(1), \ldots, \pi(i)$ have executed. Some permutations will not reach the liquidation price threshold, while others will.

Let $A$ be the liquidatable set:

$$A = \{\pi \in S_n : \exists i\ p_i(\pi) \leq p_0 - c\}.$$

where $p_i(\pi)$ is the price after executing $\Delta_{\pi(1)} \ldots \Delta_{\pi(i)}$ and $p_0 - c$ is the liquidation threshold. As we noted, some permutations don’t reach the liquidation threshold.

The liquidation payoff function is:

$$f(\pi) = \mathbf{1}_A(\pi)$$

Where $\mathbf{1}_A(\pi)$ is an indicator function that equals one if $\pi$ is in set $A$ and 0 otherwise, intuitively, it’s like choosing the permutations that liquidate the position.

Generally, we can use liquidations and auctions to construct any payoffs in the form of $\mathbf{1}_B$. Here

$$f(\pi) = \sum_{A \subset S_n} \hat{f}(A)\mathbf{1}_A(\pi)$$

Where the summation is overall $A \subset S_n$, and $\hat{f}(A)$ are the coefficients.

So, any payoff function $f$ can be written as a linear combination of liquidation payoff functions $\mathbf{1}_A$. In this sense, the liquidation payoffs $\mathbf{1}_A$ form a basis for the set of functions $f: S_n \to \mathbb{R}$. (Similar to a basis for Vector spaces ^_^)

Fairness Functionals

Now that we have established Ordering and payoffs, we move to fairness functionals. These are ways of measuring the fairness of an ordering scheme since we might want to talk about how certain orderings are fairer than others.

Given a payoff function $f: S_n \to \mathbb{R}$, we aim to measure the “fairness” of an ordering scheme that generates a set of orderings, i.e., a set $A \subset S_n$ of valid orderings.

A global fairness function is defined as a map: $L^1(S_n) \to \mathbb{R}$, which takes a payoff of a set of transactions and returns the difference (deviation) between the maximum payoff and the average payoff.

$$\begin{aligned} \tilde{\Lambda}^+(f) &= \max_{\pi \in S_n} f(\pi) - \frac{1}{n!}\sum_{\pi \in S_n} f(\pi) = \|f\|_\infty - \frac{1}{n!}\|f\|_1 = \|f\|_\infty - \mathbf{E}[f] \\ \tilde{\Lambda}^\star(f) &= \frac{\max_{\pi \in S_n} f(\pi)}{\frac{1}{n!}\sum_{\pi \in S_n} f(\pi)} = \frac{\|f\|_\infty}{\frac{1}{n!}\|f\|_1} = \frac{\|f\|_\infty}{\mathbf{E}[f]} \end{aligned}$$

A short deviation into $L^1$ norms:

In mathematics, the $L^1$ norm is a way to measure the “size” of a function. For a function that takes in values and spits out numbers, the $L^1$ norm sums up the absolute values of the function over its entire input domain.

For example, suppose we have a function $f$ that takes in integers as input and outputs numbers. The $L^1$ norm of $f$, written $\|f\|_1$, is defined as:

$\|f\|$ = sum of $|f(x)|$ for all integer inputs $x$

Where $|f(x)|$ is the absolute value of $f(x)$.

So if $f(1) = 3$, $f(2) = -2$, and $f(3) = 5$, then the $L^1$ norm of $f$ is:

$$\|f\| = |3| + |-2| + |5| = 10$$

The $L^1$ norm captures the maximum magnitude the function $f$ reaches over all inputs.

Now, in the context of the paper, The payoff function $f$ maps permutations of the transaction set to real numbers. The $L^1$ norm $\|f\|$ sums the absolute values of $f$ over all possible input permutations.

$\|f\|_\infty$ refers to the $L^\infty$ norm, also called the supremum norm, defined as:

$$\|f\|_\infty = \max_\pi |f(\pi)|$$

So, the $L^\infty$ norm takes the maximum absolute value of $f$ over all permutations $\pi$.

$\|f\|_1$ refers to the standard $L^1$ norm, defined as:

$$\|f\|_1 = \sum_\pi |f(\pi)|$$

So, the $L^1$ norm sums the absolute values of $f$ over all permutations.

The key difference is:

The $L^\infty$ norm $\|f\|_\infty$ takes the maximum value
The $L^1$ norm $\|f\|_1$ sums all values

For some more intuition on $L_1$ norms and more, check this out.

One can think of two natural global fairness functionals: the difference between the maximum and average and the multiplicative difference, which would quantify the magnitude of the maximum payoff in terms of the average. So we have:

$\Lambda^+$ Measures additive unfairness: - Defined as the difference between the max and average payoff of $f$ over $A$, it captures the additive gap between best and typical case.
$\Lambda^\star$: Measures multiplicative unfairness: - Defined as the ratio between the max and average payoff of $f$ over $A$, it Captures the multiplicative gap between best and typical case.

We move ahead with using $\Lambda^+$ because it’s neater to work with and has some desirable properties.

So when $\Lambda^+(f) = 0$, this would mean that the difference between the maximum payoff possible and the average payoff possible is 0. We can interpret this as being perfectly fair. Maximal unfairness occurs when $f = \mathbf{1}_\pi$ for some $\pi$, saturating the upper bound. Intuitively, this payoff function $f = \mathbf{1}_\pi$ returns one only for a single “jackpot” permutation $\pi$ and 0 for all other inputs.

Now that we have defined these things in the global set of orderings, we will restrict them to the subset $A \subset S_n$ (in some, localize them to a subset of the larger space we just explored)

$$\begin{aligned} \Lambda^+(f, A) &= \tilde{\Lambda}^+(f\mathbf{1}_A) = \max_{\pi \in A} f(\pi) - \mathbf{E}[f\mathbf{1}_A] \\ \Lambda^\star(f, A) &= \tilde{\Lambda}^\star(f\mathbf{1}_A) = \frac{\max_{\pi \in A} f\mathbf{1}_A(\pi)}{\mathbf{E}[f\mathbf{1}_A]} \end{aligned}$$

In an earlier paper, “Towards a theory of MEV: CFMMs,” the authors show that $\Lambda^\star(f) = O(\log n)$, given sufficient liquidity, basically as long as there’s enough liquidity, the worst case payoff only grows logarithmically in comparison to the average case in the case of constant function market makers.

So, we now have fairness functions that measure the gap between optimal and average payoffs under a set of allowable permutations. Relating the permutation set restrictions to the complexity of the payoff function yields both upper and lower bounds on the fairness functionals. These characterize the relationship between sequencing rules, payoff complexity, and fairness guarantees.

$$\begin{aligned} \Lambda^+(f, A) &= \max_{\pi \in A} f(\pi) - \mathbf{E}[f\mathbf{1}_A] = \max_{\pi \in A} f(\pi)\left(1 - \frac{\mathbf{E}[f\mathbf{1}_A]}{\max_{\pi \in A} f(\pi)}\right) \\ &= \max_{\pi \in A} f(\pi)\left(1 - \frac{1}{\Lambda^\star(f, A)}\right) \end{aligned}$$

This defines the localized fairness functional $\Lambda^+(f, A)$ for a payoff function $f$ and a set of permutations $A$.

Specifically:

$\max_{\pi \in A} f(\pi)$ is the maximum payoff achievable over all permutations $\pi$ in the set $A$. This represents the optimal strategy for an extractor trying to maximize their profit.
$\mathbf{E}[f\mathbf{1}_A]$ is the expected value of $f$ over the uniform distribution on $A$. $\mathbf{1}_A$ is the indicator function equal to 1 for $\pi \in A$ and 0 otherwise.
Their difference $\max_{\pi \in A} f(\pi) - \mathbf{E}[f\mathbf{1}_A]$ measures the gap between the best and average cases for $f$ when restricted to $A$.

Rearranging, we find that $\Lambda^+(f, A)$ can be written in terms of $\Lambda^\star(f, A)$; we will first construct a bound for $\Lambda^\star$ and then bound $\Lambda^+$ with it. Again, bounding this means we are trying to find how fair a certain set of orderings is.

Representation Theory & Uncertainty Principles

This section introduces mathematical tools to analyze how much the maximum and average payoffs of a function $f$ can differ. If we only have black-box access to $f: S_n \to \mathbb{R}$, how can we determine if the max and expected values deviate?

We saw in the liquidation example that any payoff can be constructed as the sum of indicator functions. We want to know within this set of “basis”/indicator functions which influences the payoff function $f$.

How could we go about doing this? One way to do this is if we could magically decompose every payoff $f: S_n \to \mathbb{R}$ into a sum of indicator functions and then measure the size of the coefficients.

Recall the equation for the construction of any payoff

$$f(\pi) = \sum_{A \subset S_n} \hat{f}(A)\mathbf{1}_A(\pi)$$

Here, $\hat{f}(A)$ are the coefficients of the indicator functions, and quantifying them would tell us which subset of the Ordering contributes most to the payoff function.

We first try to establish this for the construction of payoff that we’ve already seen; since we can construct payoff functions of any kind if the form is $\mathbf{1}_B$, we explore various properties. But since this form, $\mathbf{1}_B$, does not capture all the payoff types; we then look at other methods, which let us talk about the generalized form of payoff and then explore their properties.

Fourier Transforms

A very brief look into what these are.

The Fourier Transform is a mathematical tool that decomposes a signal (or function) into its constituent frequencies. Imagine you have a complex sound wave; the Fourier Transform helps you determine what “pure tones” (sine and cosine waves) make up that complex sound.

The Fourier Transform of a function $f(t)$ is given by

$$\begin{aligned} F(\omega) &= \int_{-\infty}^{\infty} f(t)e^{-i\omega t}\,dt \\ f(t) &= \frac{1}{2\pi}\int_{-\infty}^{\infty} F(\omega)e^{i\omega t}\,d\omega \end{aligned}$$

Here, $F(\omega)$ tells you the amplitude and phase of the sine and cosine waves at each frequency $\omega$ that, when added up, reconstruct $f(t)$. The Fourier Transform takes you from the time domain (where you describe signals as functions of time $t$) to the frequency domain (where you describe signals as functions of frequency $\omega$).

Imagine you’re listening to an orchestra. Your ear receives a complex mixture of sounds from various instruments, each playing at different frequencies. The Fourier Transform is like having the superpower to instantly “hear” each individual instrument’s pure tone, separating it from the orchestra’s collective sound. Later, you can combine these unique tones to recreate the original complex sound of the orchestra.

The Fourier Transform breaks down complex signals into their basic frequency components, much like identifying individual instruments in an orchestra. It’s a way to move from the time domain, where you see how a signal changes over time, to the frequency domain, where you know what frequencies make up that signal.

Fourier Walsh Transform: The Fourier-Walsh Transform is a specialized form of the Fourier Transform that uses Walsh functions instead of the usual sine and cosine functions as the basis set. Walsh functions are piecewise constant functions that take on values of +1 or -1, making them particularly well-suited for digital or binary signals.

Although the Discrete Fourier Transform (DFT) already exists as a discrete analog of the continuous Fourier Transform. The Fourier-Walsh Transform serves a different purpose and is tailored for different kinds of data. While the DFT is a discrete analog of the Fourier Transform for sampled, continuous signals, the Fourier-Walsh Transform is a specialized discrete analog for Boolean functions and binary signals.

If $f$ were a boolean function, $f: \{-1,1\}^n \to \mathbb{R}$, then the expansion

$$f(x) = \sum_{A \subset [n]} \hat{f}(A)\mathbf{1}_A(x)$$

It is called the Fourier-Walsh transform.

Where $\mathbf{1}_A(x) = 1$ if $x_i = 1$ for all $i$ in $A$, and 0 otherwise and the $\hat{f}(A)$ are called the Fourier coefficients and quantify the influence of the set $A$ on the function value. The Fourier-Walsh transform helps us prove statements that tie the global behavior of the function $f$ to properties about the sets $A$ that it is supported on. If $A$ has a small size but a large $\hat{f}(A)$, then the variables in $A$ have an outsized influence on $f$.

We can thus interpret the relationship between $|A|$ (the size of the set) and $\hat{f}(A)$ (the magnitude of the coefficients) as seeing how flat or sharp the function is.

The Plancherel theorem for the Fourier transform has a vital significance: it relates the overall “size” of a function $f$ to the sizes of its Fourier coefficients $\hat{f}(A)$. There is a Plancherel theorem relating the $L^2$ norms of $f$ and its Fourier transform:

$$\sum_{x \in \{-1,1\}^n} |f(x)|^2 = \sum_{A \subset [n]} |\hat{f}(A)|^2$$

This means the $L^2$ norm of $f$ (its “energy”) equals the $L^2$ norm of its Fourier transform $\hat{f}$. Intuitively, if a small number of Fourier coefficients $\hat{f}(A)$ contain most of the $L^2$ norm of $\hat{f}$, then the function $f$ is mainly influenced by those sets $A$. For example, if 90% of $\sum_A |\hat{f}(A)|^2$ comes from 5 sets $A$, we can say $f$ “mostly depends” on just those five sets. In transaction orderings, the Plancherel theorem allows us to identify which sets of permutations $A$ significantly influence the payoff function $f$. If the payoff depends heavily on just a few sets of orderings $A$, we can design sequencing rules or auctions to preserve those orderings. So basically, the Plancherel theorem lets us quantify which orderings matter most for the payoff $f$. It provides a precise way to measure how much freedom is needed in the allowable orderings $A$ to capture the behavior of $f$.

We now define the boolean degree $\deg(f)$ is defined as:

$$\deg(f) = \max\{|A| : A \subset \{-1,1\}^n, |\hat{f}(A)| > 0\}$$

For boolean functions $f: \{-1,1\}^n \to \mathbb{R}$, the boolean degree $\deg(f)$ measures the size of the largest input subset that significantly influences $f$. It is defined based on the Fourier-Walsh decomposition. Large Fourier coefficients on small sets $\subseteq [n]$ indicate that $f$ depends strongly on those inputs.

$T$ measures the size of the largest set $A$ that influences $f$. We can also define the degree-$t$ part $f^{\leq t}$ as:

$$f^{\leq t}(x) = \sum_{\substack{A \subset \{-1,1\}^n \\ |A| \leq t}} \hat{f}(A)\mathbf{1}_A(x)$$

So $f^{\leq t}$ restricts $f$ to sets $A$ of size $\leq t$. The degree gives a precise way to measure how “complex” a boolean function is.

Now that we have established all the properties we are interested in for the indicator function, we look at more generalized payoffs.

The symmetric group $S_n$:

This contains all possible permutations (orderings) of $n$ elements.
For transaction orders, $S_n$ would be all possible orderings of $n$ transactions.
The size of $S_n$ is $n!$ (factorial of $n$).

In the case of abelian groups, indicator functions can often be represented as vectors because the group operation is commutative. However, the non-commutativity complicates things for non-abelian groups like the symmetric group. Here, a more complex representation is needed, and that’s where matrices come in.

Therefore, instead of having indicator functions $\mathbf{1}_A: \{-1,1\}^n \to \mathbb{R}$, we have representations. A representation of a group $G$ is a way to represent the elements of $G$ as linear transformations on a vector space $V$. Intuitively, it shows how $G$ acts on $V$.

A representation $\rho$ of $S_n$:

$$\rho : S_n \to \mathsf{GL}(V^d)$$

$\mathsf{GL}(V^d)$ is the general linear group of invertible matrices acting on a vector space $V$ of dimension $d$.

Irreducible representations are the basic building blocks - they cannot be decomposed into smaller representations. It is a fact that finite groups have a finite number of irreducible representations. Representations will serve as the analog of indicator functions for non-abelian groups.

The Fourier transform and representation theory allows analyzing functions on non-commutative groups like $S_n$ analogously to Fourier analysis of processes on $\mathbb{R}^n$. The Fourier transform $\hat{f}(\rho)$ of a function $f: S_n \to \mathbb{R}$ is given by

$$\hat{f}(\rho) = \sum_{\pi \in S_n} f(\pi)\rho(\pi)$$

The Fourier transform decomposes a function $f: S_n \to \mathbb{R}$ into irreducible representations. The $\lambda$th Fourier coefficient $\hat{f}(\rho^\lambda)$ is a matrix capturing how $f$ acts on $V_\lambda$. Note that this is a sum of matrices.

As we saw earlier, we have a Plancherel theorem in this case, too, defined by

$$\|f\|_2 = \frac{1}{n!}\sum_{i=1}^{k} \dim(\rho_i)\,\mathbf{Tr}[\hat{f}(\rho_i)^* \hat{f}(\rho_i)]$$

To define the boolean degree, we need an analog, though. Ultimately, the boolean degree in the indicator function case gave us an idea of how much influence the largest subsets of inputs had on $f$. In this case, we need a better way of talking about the representations of $S_n$.

Every permutation in $S_n$ can be decomposed into cycles. These cycles correspond to partitions of $[n]$, and each partition describes an irreducible representation of $S_n$. The function $f$ is expanded using the inverse Fourier transform. The sum involves traces of matrices corresponding to irreducible representations, weighted by some degree $d$.

$$f(\pi) = \frac{1}{n!}\sum_{\lambda \vdash n} d_\lambda\,\mathbf{Tr}[\hat{f}(\rho^\lambda)\rho^\lambda(\pi)] = \frac{1}{n!}\sum_{\lambda \vdash n} d_\lambda f^{=\lambda}(\pi)$$

Where $\mathbf{Tr}$ is the matrix trace and $d_\lambda = \dim(V_\lambda)$ is the dimension of $\rho^\lambda$.

It turns out that the set of partitions of $[n]$ serves to index the irreducible representations of the symmetric group $S_n$. The representations $\rho$ are indexed by partitions $\lambda$ of $n$. This is a way to express functions on $S_n$ in terms of invariant basis functions under certain permutations. This set is called the Specht module and allows one to decompose $S_n$.

$$L^1(S_n) = \bigoplus_{\lambda \vdash n} S^\lambda$$

Thus, the sum in the equation (in the case of expanded $f$) represents the projection of $f$ to each Specht module.

All of this helps us ultimately to define the boolean degree of this function $f$, as:

$$\deg(f) = \min\{n - \lambda_1 : \lambda \vdash n, |f^{=\lambda}| > 0\}$$

Where $\lambda \vdash n$ means $\lambda$ is a partition of $n$. Where $\lambda_1$ is the largest part of $\lambda$, this measures the complexity of $f$ on $S_n$.

So, in summary, the Fourier analysis generalizes to non-abelian groups like $S_n$ while still providing ways to analyze the complexity of functions via representations and boolean degrees. The same high-level goals apply regarding quantifying the influence of sets and relating degrees of $f$ and $\hat{f}$.

Let’s imagine a small example:

Suppose we have $n = 3$ transactions in our blockchain: A, B, C.

The symmetric group $S_3$ would contain all possible orderings of these three transactions:

$$S_3 = \{ABC, ACB, BAC, BCA, CAB, CBA\}$$

So $S_3$ has size $3! = 6$.

Now suppose our payoff function $f$ assigns a profit to each Ordering:

$f(ABC) = 6$, $f(ACB) = 2$, $f(BAC) = 3$, $f(BCA) = 4$, $f(CAB) = 2$, $f(CBA) = 5$

If we take the Fourier transform of $f$ over $S_3$, it will decompose $f$ into frequency components based on cycle types:

There are three cycle types:

3-cycles: CAB, BCA
2-cycles: ACB, BAC, CBA
1-cycles: ABC

The transform would reveal how much each cycle type contributes to the payoff $f$:

$f(\text{3-cycle}) = 3$ (average of CAB, BCA payoffs)

$f(\text{2-cycle}) = 4$ (average of ACB, BAC, CBA)

$f(\text{1-cycle}) = 6$ (just ABC)

Here, we see the 1-cycle orderings rely mostly on specific orders, while 3-cycles don’t rely on order much.

In this case, the boolean degree $\deg(f)$ would be one since the full payoff relies on a 1-cycle (specific order matters).

A cycle refers to how many elements are permuted or swapped in the Ordering.

So in $S_3$:

A 1-cycle means only one element is being moved.
A 2-cycle means two elements are swapped. Like ACB, where C and B are swapped compared to ABC.
A 3-cycle means all three elements are permuted. Like BCA, where A, B, and C are all moved compared to ABC.

In general, an $k$-cycle in $S_n$ means $k$ of the $n$ elements have their positions changed or cycled.

So, in our example:

1-cycle orderings like ABC rely heavily on that specific order to achieve the payoff. A small change breaks it.
3-cycle orderings like BCA are not dependent on the order much. You can change the order and still get similar payoff.

The Fourier transform shows the 3-cycle orderings contribute a moderate amount to the payoff $f$, while the 1-cycle contributes the most. This shows the payoff relies on one specific ordering.

As we saw earlier, Uncertainty principles relate a function $f$ to properties of its Fourier transform $\hat{f}$. They state that $f$ and $\hat{f}$ cannot both be highly concentrated or “spiky” - there is an inherent trade-off, i.e., they cannot be localized simultaneously. The paper mentions, “One can also view these as saying if $f$ is concentrated on a small set $S$ of values, then $\hat{f}$ cannot be concentrated on a set of values more than some decreasing function.”

Now we look at the following equation, which connects the norm of a function and its Fourier transform:

$$\frac{\|f\|_1}{\|f\|_\infty} \frac{\|\hat{f}\|_1^{(S)}}{\|\hat{f}\|_\infty^{(S)}} \geq |G|$$

This relates to the $L^1$ norms of $f$ and $\hat{f}$, measuring their concentrations.

If $G = S_n$,

$$\frac{\|f\|_1}{\|f\|_\infty} \frac{\|\hat{f}\|_1^{(S)}}{\|\hat{f}\|_\infty^{(S)}} \geq n! \implies \frac{1}{\tilde{\Lambda}^\star(f)} = \frac{\frac{1}{n!}\|f\|_1}{\|f\|_\infty} \geq \frac{\|\hat{f}\|_\infty^{(S)}}{\|\hat{f}\|_1^{(S)}}$$

The significance of this inequality is:

It upper bounds the gap between the max and average value of $f$ in terms of $\hat{f}$
If $\hat{f}$ is spread out, the gap is small, meaning randomness provides good fairness
If $\hat{f}$ is concentrated, the gap can be large, so complex rules are needed for fairness
The degree $s$ of $f$ appears in $k(n, s)$, relating the complexity of $f$ to the required freedom in ordering rules

So, in essence, this uncertainty principle allows translating the complexity of a payoff function $f$ to guarantee the fairness of random/limited orderings in terms of properties of its Fourier transform $\hat{f}$.

t-intersecting sets of permutations

The goal is to formalize the notions of “small” and “large” sets of permutations $A$ about the payoff function $f$. This is done using the concept of $t$-intersecting sets - sets $A$ where every pair of permutations in $A$ share at least $t$ pairwise elements that are mapped to the same points. Formally, $A$ is $t$-intersecting if for all $\pi, \pi'$ in $A$, there exist $t$ pairs $(i_1, j_1), \ldots, (i_t, j_t)$ with $\pi(i_k) = \pi'(i_k) = j_k$.

Recent results show $t$-intersecting sets have size bounded by $(n-t)!$ [KLMS23]. Moreover, if $|A| = \Omega((n-t)!)$, fixed points are shared by all $\pi$ in $A$. This implies indicator functions $\mathbf{1}_A$ for such $A$ have degree $\geq t$.

The $t$-intersecting sets concept is relevant for “fair ordering” protocols that validators use to agree on a set of valid transaction orderings $A$. These protocols work by having validators reach a consensus on a directed acyclic graph (DAG) $G$ representing order precedence. A valid ordering is any topological sort of $G$. However, issues like the Condorcet paradox can still arise.

This results in cycles in $G$ that are conserved across all topological sorts. So the set of valid orderings $A$ ends up being $t$-intersecting for some minimal $t < n$, as orderings keep some cycles fixed.

The value of $t$ depends on the level of agreement among validators:

$t$ close to 0 means validators had nearly evenly split votes on order for most transactions
$t$ close to $n$ means almost total agreement on a full precedence ordering

So, $t$ quantifies the order flexibility resulting from the fair ordering protocol.

In summary, $t$-intersecting sets are a natural model for the constraints on orderings from fair ordering protocols, allowing the application of the fairness results we now see.

Main results

There are two key results - an upper and lower bound on the fairness functionals for a payoff function $f$ and set of orderings $A$.

Upper bound: If $A$ is $t$-intersecting and $t \geq \deg(f)$, there is a non-trivial upper bound on the unfairness:

$$\Lambda^+(f, A) = \|f\mathbf{1}_A\|_\infty - \frac{1}{n!}\|f\mathbf{1}_A\|_1 \leq \left(1 - \frac{C}{\binom{n}{s}^2 c^{\sqrt{s}} s!}\right)\|f\mathbf{1}_A\|_\infty$$

Where $s = \deg(f)$. This means randomness provides some fairness if $A$ has sufficient overlap relative to $f$’s complexity.

Lower bound: If $t < \deg(f)$ and $|A|$ is sufficiently large, there is a non-trivial lower bound:

$$\Lambda^+(f, A) = \|f\mathbf{1}_A\|_\infty - \frac{1}{n!}\|f\mathbf{1}_A\|_1 \geq \left(1 - \frac{c'(s - t - 1)}{(n - t)!}\right)\|f\mathbf{1}_A\|_\infty$$

This means that if $A$ omits critical orderings, significant unfairness is guaranteed.

Together, these relate worst-case unfairness to the interplay of $t$, $\deg(f)$, and $|A|$.

So, $t$-intersecting sets “bandlimit” functions to degree $\geq t$. The degree of $f$ relative to $t$ now allows formalizing “small” and “large”:

If $t < \deg(f)$, $A$ is “small” and may omit critical orderings → lower bound
If $t \geq \deg(f)$, $A$ is “large” and captures the complexity of $f$ → upper bound

So, $t$-intersecting sets allow connecting the size and overlap of $A$ to the complexity of $f$ via degrees.

The notion of a “bandlimit” arises from signal processing and Fourier analysis. In that context, bandlimiting refers to limiting a function’s Fourier transform to only low frequencies below some threshold. In the context of this work on transaction orderings, the idea of bandlimiting comes up when considering $t$-intersecting sets of permutations $A$.

Specifically:

$t$-intersecting sets $A$ have the property that permutations in $A$ share at least $t$ fixed point pairs $(i, j)$ ($t$ transactions for our understanding)
This means $A$ limits permutations to those with sufficient overlap in their structure. (maybe a fair ordering)
In terms of Fourier analysis on the symmetric group $S_n$, this restriction ends up filtering out representations indexed by partitions $\lambda$ with $\lambda_1 > n - t$
So $A$ only includes “low degree” Fourier modes of degree $< t$

In signal processing, this is analogous to low-pass filtering - allowing low frequencies to pass while blocking high frequencies. Hence, $t$-intersecting sets effectively “bandlimit” functions on $S_n$ by restricting permutations to those sharing $t$ fixed points. This band-limiting property connects the overlap of $A$ to the complexity of payoff functions $f$. If $\deg(f) < t$, $A$ omits critical orderings needed to capture $f$. If $\deg(f) \geq t$, $A$ retains enough overlap to represent $f$. So, in summary, $t$-intersecting sets provide a way to bandlimit or restrict complexity in a precise way connected to Fourier analysis on $S_n$. This is then leveraged in the main proofs bounding fairness functionals.

Implications on MEV:

Simple payoff functions like CFMMs have low degree → simple sequencing gives good fairness/high welfare
Complex payoffs like liquidations have a high degree → complex rules needed for fairness
Like a Nyquist sampling rate theorem for fairness

Implications on Fair Ordering:

If $t < \deg(f)$, unfairness lower bound applies → paradoxes cause unfairness
If $t \geq \deg(f)$, upper bound applies → high agreement gives fairness

Overall, the bounds clarify how properties of $A$ interact with the complexity of $f$ to limit fairness.

Conclusion and broad idea

The paper’s conclusion is as follows: “Consensus-enforced ordering rules need to be constructed on an application-level basis.” this follows from the fact that, as we saw, different payoff functions have different degrees, so they need different sequencing rules to achieve high welfare/fairness.

We can think of this paper as giving sampling rules:

The payoff function $f$ is like the continuous signal. It has varying levels of complexity.
The Fourier transform $\hat{f}$ represents the frequency decomposition. Boolean degree = highest freq.
Restricting to $A$ is like sampling the permutation space. $t$-intersecting is like the sampling rate.
If $t \geq \deg(f)$, it’s like sampling above the Nyquist rate. Capture complexity gives a fairness guarantee.
If $t < \deg(f)$, it’s like under-sampling. Fails to capture full complexity, allows unfairness.
So, in essence, $t$ relative to $\deg(f)$ is like the sampling rate close to signal frequency content.

Look at this talk by Tarun Chitra for more.

On Privacy and CFMMs

Tue, 18 Apr 2023 18:13:16 +0000

Constant Function Market Makers (CFMMs) have emerged as an innovation in decentralized finance, providing an efficient and computationally cheap on-chain solution for exchanging digital assets without needing traditional order books. CFMMs revolutionize how assets are traded, enabling a more seamless and accessible marketplace and have become the most widely used decentralized crypto product. However, as the popularity of decentralized finance grows, so does the need to address the privacy concerns arising from the transparent nature of blockchain-based transactions. In this article, we will delve into the nature of privacy in CFMMs and discuss them in reference to the works “A note on privacy in Constant Function Market Makers” by Angeris et al. and “Differential Privacy in Constant Function Market Makers” by Tarun Chitra et al.

Why Privacy? CFMMs offer LPs looking for a passive yield to offer their assets to willing traders. This works in CFMMs by ensuring that the “trading function” is kept constant, i.e. the price is determined using this function. Privacy is primarily desired on three ends so:

LPs don’t get adversely selected against
Users have all their trades publicly visible to everyone and directly being associated with their identity.
The public nature of transactions, of course, also allows third parties to front-run a user’s trades.

For example, In constant sum market makers, the quote is always a single price at all times so an adversary cannot discern if the trade is of size 50 or 50 trades of size 1, but the LPs get adversely selected against because anyone can buy large quantities while this price is decreasing in other places. Designing better CFMMs involves quantifying and qualifying the tradeoffs between the ideal privacy we desire from our CFMMs and the adverse selection that an LP can potentially face.

Although smart contracts that utilize Zero Knowledge proof systems should be able to execute CFMM transactions privately. It has been shown in recent work, “Why cant you build a private uniswap with ZKPs”, that informally and “heuristically”, the timing of a trade implicitly leaks privacy in Uniswap and other CFMMs. In simple words, the timing of the trade can help you reconstruct the values.

Note of Privacy in CFMMs

This paper shows that:

Even under relatively weak adversaries, CFMMs are generally unable to preserve privacy
Provide some mitigation mechanisms and start a discussion around them to help improve user privacy

Constant Function Market Makers: A CFMM is an automated market maker defined by its reserve quantities and trading function. The reserve quantities $R$ and a trading function $\psi$. Ultimately, the intuitive notion of a CFMM is very simple; an agent/user proposes a trade $\Delta$ which is a vector signifying trade quantities. While in the original CFMM paper, there are two different tuples/vectors for inputs and outputs. In this case, we denote $\Delta_i$ as a positive quantity if it was given to the CFMM, and $\Delta_i$ is negative if it is taken. The CFMM checks if the trade satisfies the trading function, and the reserves are then updated if this is deemed valid. If you aren’t entirely familiar with CFMMs, you can take a look at this.

$$\psi(R + \Delta) = \psi(R),$$

We assume that the trading function $\psi$ is strictly concave, which it is for most AMMs of interest except for constant sum market makers and other special cases. The marginal price for a fee-less CFMM is given by

$$\nabla\psi(R) = \lambda c,$$

where $\lambda > 0$.

The Attack

Assume that Eve is the adversary trying to discover the quantity traded by Alice. Eve is unable to see the exact quantities traded by Alice but knows when Alice’s transaction took place.

Eve is able to query the marginal price of the CFMM at the current reserves and whether a given trade $\Delta$ is valid. Eve can also query the CFMM before and after the transaction. This adversary attack fails if the transaction time is obfuscated, but such protocols don’t exist today.

Before we talk about the sequence of the attack, it is important to understand that Eve can always reconstruct the reserve amounts given the marginal price at the current system reserves and a single nonzero feasible trade. Once this is obtained, it is enough to compute the reserve amounts before and after Alice’s trade to recover the traded amounts.

The sequence of the attack is as follows:

Eve queries the marginal price of CFMM at current reserves to get some vector $c$ and then queries any valid nonzero trade $\Delta$ is not equal to 0.
Knowing the functional form of the trading function, $\psi$, Eve solves the following and finds a solution to find $R$ (the reserves)

$$\nabla\psi(R) = \lambda c, \quad \psi(R + \Delta) = \psi(R).$$

Let $\Delta a$ be Alice’s trade, and the new reserves be $R_a = R + \Delta a$ (This is not known to Eve, but the CFMM can now be queried in this new state)
After the trade $\Delta a$, Eve queries the contract again to get a new marginal price $c'$ and then queries another nonzero trade $\Delta'$. She can solve the following system of equations to obtain the reserves $R_a$.

$$\nabla\psi(R_a) = \lambda c', \quad \psi(R_a + \Delta') = \psi(R_a),$$

Finally, we do $R_a - R = \Delta_a$, which are Alice’s traded values.

Reserve Discovery in Uniswap

Ultimately following this proof and plugging in the respective equations for uniswap, we find that only by having the marginal price $c$ and a nonzero trade $\Delta$, we can recover the uniswap reserves. While this is a special case, it is possible to reconstruct reserves using any two nonzero, distinct feasible trades.

Future extensions of the proof

When the function has non-zero fees, the CFMM must satisfy

$$\psi(R + \gamma\Delta_+ - \Delta_-) = \psi(R),$$

where $\gamma$ is the fees.

Unknown marginal price - If in the case that the marginal price is unknown and cannot be accessed directly, Eve can compute an approximate solution by performing $n$ queries.

Mitigating Strategies

Now that we have understood that providing privacy directly in CFMMs is impossible as long as the timing of the trade is emitted, i.e. known, We’ll talk about a few ways in which we can think about the modifications we can do to provide some modicum of privacy.

Randomness in price - Eve requires knowledge of $c$ in order to reconstruct the reserves, i.e. knowledge of the marginal price. Adding some amount of randomness can help prevent Eve from reconstructing the reserve values. Controlling randomness, in this case, is very difficult because it needs to be consistent, and any pattern can probably be deciphered by querying multiple trades. And generally, as we know that if the difference, i.e. randomness, is large, it will quickly get exploited by arbitrageurs resulting in additional loss of liquidity for the LPs. In simple words, randomness in price forces worse prices on the end users.
Batching Orders - The CFMM waits till several trades $\Delta$ are accepted and updates its reserves only after all trades are executed. Batching orders has a tradeoff against the performance of the system and has an immediate degrading effect on the user experience. The CFMM also has to ensure that all the trades are not by Eve (except Alice’s) because, in that case, it would reveal the reserves before and after the batch of trades, hence giving a much simpler way of getting the values Alice traded. In simple words, batched orders add latency to the end users.

Differential Privacy in CFMMs

Now that we’ve established what we expect from CFMMs when we talk about privacy, the issues we face and some of the mitigation strategies. At a high level, privacy in CFMMs is as simple as preventing anyone from figuring out the trade prices of a user. A natural question to ask is, what kind of tradeoff is possible between privacy and pricing? What kind of guarantees can we give when we design a CFMM with privacy-first attributes? And what are ways we can achieve this?

In the paper “Differential Privacy in CFMMs”, Tarun Chitra et al. the authors sought to answer two major questions:

What is the minimum number of samples $n(\delta)$ such that an adversary is unable to infer the actual sizes of the trades beyond a precision $\delta$? i.e. in other words, what should be the size of a batch?
How much worse is the worst price offered to a user via such a mechanism? i.e. in other words, in a batch, what’s the worst trade execution price looking like?

The paper establishes the following:

Quantifies the trade-off between pricing and privacy in CFMMs
Analyze a simple privacy mechanism called Uniform Random Execution and prove that it provides $(\epsilon, \delta)$ differential privacy, where $\epsilon$ depends on the curvature of the CFMM/the number of trades executed.
Conclude that when it comes to CFMMs with non-zero curvature, one cannot perform better than Uniform random execution when it comes to providing privacy.

Some Background Knowledge

Before we look at various aspects of the paper, we’ll take a look at some definitions again:

CFMM State: The reserves of a CFMM are updated using the following formula where $\Delta$ is transferred from the trader to the dex, and $\Lambda$ is transferred from the DEX to the trader. The pair of $(\Delta, \Lambda)$ make up the “proposed trade”, which is initiated by a user. $\Delta$ signifies the tender basket, and $\Lambda$ signifies the received basket.

$$R^+ = R + \Delta - \Lambda,$$

Trading Function: As we saw earlier in this article, a proposed trade $(\Delta, \Lambda)$ is only accepted when

$$\psi(R + \gamma\Delta - \Lambda) = \psi(R)$$

in the earlier section of the article, we only saw an $R + \Delta$ instead of denoting the tender and received basket separately. This is also because here, we consider fees explicitly. Also, the term $\gamma$ (<1) signifies the trading fee. The term “constant function” for a CFMM comes from the above condition, that regardless of a trade being executed, the function remains constant when executed/examined.

Something to note about the trading function again, as we did earlier, is we assume $\psi$ is strictly concave. Quickly taking as an example,

The trading function for Uniswap generally is

$$\psi(R) = R_1 R_2$$

but this is neither convex nor concave, but it can be equivalently written as

$$\psi(R) = \sqrt{R_1 R_2},$$

which is strictly concave in nature whenever $R > 0$.

Curvature: The marginal price for a trade size of $\Delta$ for a CFMM is:

$$g(\Delta) = -\frac{\partial_1 \psi(R, R', \Delta, \Delta')}{\partial_2 \psi(R, R', \Delta, \Delta')}$$

Here $\partial_1$ and $\partial_2$ are the partial derivatives wrt to the ith argument. $g$ is known as the price impact function, as it represents the final price (marginal price) of a trade.

There are two CFMM properties that we care about, $\mu$-stable and $\kappa$-liquid. They are satisfied if

A CFMM is $\mu$-stable if

$$g(0) - g(-\Delta) \leq \mu\Delta$$

A CFMM is $\kappa$-liquid if

$$g(0) - g(-\Delta) \geq \kappa\Delta$$

Basically, $\mu$-stable means whenever a non-negative trade size of $\Delta$ does not change the market price by more than $\mu\Delta$, it places a linear upper bound on the maximum price impact that a bounded trade can have. Similarly, $\kappa$-liquid means that there is some price slippage when a token is sold but is linearly bounded from below by at least $\kappa\Delta$.

Differential Privacy

A small digression to understand where differential privacy comes from and what the entire progression towards it was like:

Data anonymization: Initially, the primary approach to protect privacy was to remove personally identifiable information (PII) from datasets. However, several re-identification attacks demonstrated that anonymized data could be linked back to individuals, leading to privacy breaches. This would look like removing name, address etc.

What personal identifiable information looks like

The progression after that

K-anonymity: To address the limitations of data anonymization, k-anonymity was proposed. In a k-anonymized dataset, each record is indistinguishable from at least k-1 other records with respect to certain attributes. However, k-anonymity has its limitations, as it does not protect against attacks based on the distribution of sensitive attributes.
L-diversity: L-diversity extends k-anonymity by ensuring that each group of records sharing the same attributes also contains at least “l” distinct values for the sensitive attributes. This provides stronger privacy guarantees but still has limitations in protecting against attacks that exploit background knowledge.
T-closeness: To further strengthen privacy, t-closeness was introduced, which requires the distribution of sensitive attributes in each group of records to be close to the overall distribution of those attributes in the entire dataset. This makes it more difficult to infer an individual’s sensitive attributes based on their non-sensitive attributes.
Differential privacy: Differential privacy emerged as a more robust privacy framework that mathematically guarantees an individual’s privacy. It achieves this by adding noise to data queries, ensuring that the presence or absence of any individual data point does not significantly change the outcome. This makes it extremely difficult for adversaries to infer sensitive information about individuals from the data, even with background knowledge.

Let’s understand differential privacy with the help of an example. Let’s say you have a group of friends who want to know the average age of the group, but no one wants to reveal their exact age. To solve this problem while maintaining privacy, you can use a simple form of differential privacy.

Each person adds random noise (e.g., a random number between -5 and 5) to their age. This altered age is called the “noisy age.”
Everyone shares their noisy age with the group.
The group calculates the average of the noisy ages.
To get the actual average age, the group subtracts the average noise that was added in Step 1.

Now, the group has calculated the average age without revealing any individual’s exact age. Even if someone tries to guess a person’s age from their noisy age, it would be difficult because of the random noise added to it.

Ultimately privacy frameworks like this sort of tried to anonymize information which helped obscure data which could personally identify someone but the aggregate statistics of the dataset remained somewhat similar.

One way of guaranteeing differential privacy is $\varepsilon$-differential privacy (which was actually introduced in the original paper. More methods have come out since then, but most are relaxations of the $\varepsilon$ condition. Let $\varepsilon$ be a positive real number and $A$ be a randomized algorithm that takes a dataset as input (representing the actions of the trusted party holding the data). The algorithm $A$ is said to provide $\varepsilon$-differential privacy if, for all datasets $D_1$ and $D_2$ that differ on a single element (i.e., the data of one person), the following condition holds:

$$Pr(A(D_1) \in S) \leq \exp(\epsilon) \cdot Pr(A(D_2) \in S)$$

where

$Pr[A(D_1) \in O]$ is the probability that algorithm $A$ produces an output in the set $S$ when applied to dataset $D_1$.
$Pr[A(D_2) \in O]$ is the probability that algorithm $A$ produces an output in set $S$ when applied to dataset $D_2$.
$\varepsilon$ (epsilon) is a non-negative value representing the privacy loss parameter. Smaller $\varepsilon$ values correspond to stronger privacy guarantees.

In other words, $\varepsilon$-differential privacy ensures that the presence or absence of any individual’s data does not significantly change the probability distribution of the algorithm’s output.

$(\epsilon, \delta)$ differential privacy is a relaxation of the standard definition of differential privacy, known as $\varepsilon$-differential privacy. The $(\epsilon, \delta)$ differential privacy provides a more flexible privacy guarantee by introducing an additional parameter $\delta$. It allows for a small probability of a slightly larger privacy breach. A randomized algorithm $A$ is said to provide $(\epsilon, \delta)$-differential privacy if, for all datasets $D_1$ and $D_2$ that differ on a single element (i.e., the data of one person), the following condition holds:

$$Pr(A(D_1) \in S) \leq \exp(\epsilon) \cdot Pr(A(D_2) \in S) + \delta$$

Where the terms are the same as the above with just the addition of a larger deviation term $\delta$.

Problem Construction

As we saw earlier towards the end of discussing the first paper, there are two ways to mitigate the attack model by eve:

Randomizing Price
Batching orders

We now discuss the rest of the work in “Differential privacy in CFMMs” by Tarun Chitra et al., The paper presents a threat model (which is the eve model from the earlier paper) and then constructs two solutions and talks about various aspects of the solutions and the bounds that we might see.

The threat model: Eve attempts to discover the quantities that Alice has traded. In this model, we go further and assume that Eve wants to discover all the quantities by a set of agents. Eve is unable to see the quantities but knows when the traders’ transactions $\Delta_1, \Delta_2, \ldots, \Delta_n$ are executed. Eve does not know the order. The goal is to estimate the order and sizes. Eve’s only ability is to interact with the CFMM to see if a trade is valid and compute the marginal spot (which is done through accessing the state of the CFMM before and after the transactions of some trader in our earlier case Alice).

SURE - Sample Uniform Random Execution

One of the simple ways to introduce randomness into a CFMM is to randomly permute the set of trades to be executed.

Suppose we are given a vector of valid trades,

$$\Delta_1 \in \mathcal{A}_\varphi(R), \quad \Delta_i \in \mathcal{A}_\varphi\left(R + \sum_{j=1}^{i-1} \Delta_i\right)$$

For a trade vector $\Delta$, the above condition will be referred to as $\mathcal{A}_\varphi(\Delta)$.

The sure mechanism draws a random permutation

$$\pi \sim_{\mathrm{Unif}} S_n$$

and constructs a sequence of trades:

$$\Delta_i^\pi = \Delta_{\pi(i)}$$

Now consider the original trades marginal prices $p_1, p_2, p_3 \ldots p_n$ and the permuted price $p^\pi_1, p^\pi_2 \ldots p^\pi_n$, we would want to bound the maximum deviation between the true price of a trade $p$ and the permuted price $p^\pi$. What does this mean? Let’s just say the original trade prices were [10,20,30,40 ….] and the permuted order trade execution yields the prices [40, 30, 10, 20….]. When we consider the trades in the first place, the trade price is now 4x worse for the user, so when constructing any such ordering, it is paramount to bound this maximum deviation. This maximum deviation is given as follows:

$$\mathcal{E}_{SURE} = \mathop{\mathbf{E}}_{\pi \sim S_n}\left[\max_{i \in [n]} |p^\pi(i) - p(i)|\right]$$

Mentioning again, this deviation, i.e. quantity, corresponds to the bound on the worst quoted price that a trader would get. We also want to capture and discuss the difficulty that the adversary would face learning/figuring out the values of $\pi$ chosen given only the prices $p^\pi_n$.

Let’s consider two scenarios where in we can talk about the deviation impact.

One, if all the trade sizes are unique, then beyond some precision, it is very difficult to compute the original trade order since every permutation would be unique. What could the worst price impact look like? In this case, SURE would work pretty well, and the maximum deviation would be bounded since the probability of having a long list of trades in the same direction is very low (i.e. only buy or only sell).

Two, now consider that the subset of trades is similar, i.e. actually the same value. For example that in n trades, the first trade is of the size 100, and all the other trades are of the size 1. When doing this in the original order, everyone gets the adjusted price after the big trade, while in any other permutation, every trade that gets executed before the large trade of size 100 gets a better price, and every one after it gets a worse price than before. Therefore, in this case, SURE requires that the trade distribution should have sufficient entropy and the distribution of trades shouldn’t be too concentrated (as we saw, a large trade makes the randomization give a worse price to everyone after and a better one to everyone before). We discussed the two properties that we care about when it came to CFMMs earlier.

Note: Skip to Uniform Random Execution and just read the paragraph before it if you don’t want to delve into the mathematical treatment of finding the bounds

A CFMM is $\mu$-stable if

$$g(0) - g(-\Delta) \leq \mu\Delta$$

A CFMM is $\kappa$-liquid if

$$g(0) - g(-\Delta) \geq \kappa\Delta$$

We will try to bound the maximum of the price process through the use of these and random binary trees. We analyse SURE on a subset of the total set of allowable input trades. Suppose the price impact function $g$ is $\mu$-stable and $\kappa$-liquid on an interval $[-M, M]$. This implies

$$\sum_{j=1}^{i} \kappa\Delta_{\pi(j)} - \mu\Delta_j \leq p^\pi(i) - p(i) \leq \sum_{j=1}^{i} \mu\Delta_{\pi(j)} - \kappa\Delta_j$$

which then bounds the partial sums of permuted trades by

$$\kappa\,\mathbf{E}\left[\max_i \left|\sum_{j=1}^{i} \Delta_{\pi(j)} - \frac{\mu}{\kappa}\Delta_j\right|\right] \leq \mathbf{E}\left[\max_i |p^\pi(i) - p(i)|\right] \leq \mu\,\mathbf{E}\left[\max_i \left|\sum_{j=1}^{i} \Delta_{\pi(j)} - \frac{\kappa}{\mu}\Delta_j\right|\right]$$

We define the partial sum as

$$R_i(\Delta, \pi) = \sum_{j=1}^{i} \Delta_{\pi(j)} - \frac{\mu}{\kappa}\Delta_j$$

and construct a binary search tree $T$, whose root is $R_1$, that is just the partial sum of the first element, then $R_2$ is the partial sum of two elements, $R_3$ is the partial sum of 4 elements etc. An example of a binary tree can be seen below:

This representation of partial sums as a tree provides a natural geometric description of the maximum price deviation. The $\max_i R_i(\Delta, \pi)$ is a lead node in this tree. By using curvature and tree structure, the problem of maximum price deviation (which is a continuous problem) is turned into a combinational one (regarding a random search tree). This modifies our earlier bounds to

$$\begin{aligned} \max_i |p^\pi(i) - p(i)| &\leq \mu\left(|R_1(\boldsymbol{\Delta}, \pi)| + \max_j \left|\Delta_{\pi(j)} - \frac{\kappa}{\mu}\Delta_j\right| \cdot \mathsf{height}(T(\mathbf{R}(\boldsymbol{\Delta}, \pi)))\right) \\ \max_i |p^\pi(i) - p(i)| &\geq \kappa\left|R_1(\boldsymbol{\Delta}, \pi) + \min_j \left(\Delta_{\pi(j)} - \frac{\mu}{\kappa}\Delta_j\right) \cdot \mathsf{height}(T(\mathbf{R}(\boldsymbol{\Delta}, \pi)))\right| + O(1) \end{aligned}$$

We know from the paper “The Height of a random binary search tree” by Reed et al. 2003, that if a random binary search tree has unique elements, then the expected average height is given by the following, where $\alpha \approx 4.31107$ is the unique solution on the interval $[2, \infty)$ of the equation $\alpha \ln((2e/\alpha)) = 1$ and $\beta = \frac{3}{2\ln(\alpha/2)}$ and ln means natural log. The variance of the height of the tree is constant.

$$\alpha \ln n - \beta \ln \ln n + O(1)$$

Plugging this in the bounds we found earlier, we find that,

$$\Delta_{\min} = \left|\min_{i,j} \Delta_i - \frac{\mu}{\kappa}\Delta_j\right| = \Omega(1)$$

and

$$\Delta_{\max} = \left|\max_{i,j} \Delta_i - \frac{\kappa}{\mu}\Delta_j\right| = O(1)$$

This implies that an adversary cannot determine a trade size with precision greater than $\Omega(\kappa)$ since there is always a minimum price discrepancy of $\Omega(\kappa \log n)$. On the other hand, the upper bound on the price deviation means that the mechanism will not cause too great a price impact for users. There are some assumptions here that we now note. We assume that $R_j(\Delta, \pi)$ is equiprobable, but this doesn’t hold true when one of the trades is much larger than all other trades. And the average height is only used when all the elements of the binary tree are unique. This means that for SURE to work:

All permutations of partial sums should be unique.
$\mu \leq (\max_i \Delta_i)\kappa$
And, No trade should massively outsize all others.

Uniform Random Execution

To solve this, The authors propose Uniform random execution by adding two things to the SURE algorithm

By adding noise independent of $\Delta, \mu, \kappa$
Splitting trades

The Uniform Random Execution mechanism is governed by three parameters

$c_{\min}$: Lower bound on $\Delta_{\min}$ i.e the minimum price deviation
$s$: Split threshold that controls the average chunk size for a big trade
$k$: Multiple of $(1+s)\Delta_{\min}$ that requires splitting

To guarantee the following,

$$\Delta_{\min} = \left|\min_{i,j} \Delta_i - \frac{\mu}{\kappa}\Delta_j\right| = \Omega(1)$$

We add Laplace noise. We do this by constructing random variables $\xi_1, \ldots, \xi_n$ drawn i.i.d (independent and identically distributed) from a distribution that can depend on a particular $\Delta$ but guarantees that $\Delta = \Delta + \xi$.

$$\Delta_{\max} = \left|\max_{i,j} \Delta_i - \frac{\kappa}{\mu}\Delta_j\right| = O(1)$$

The upper bound, as seen above, can be reduced by splitting trades. This reduces the max and also increases the privacy of SURE. Instead of splitting trades in two, we split trades into $m(\Delta_i)$ pieces, where $m(\Delta_i)$ is defined as

$$m(\Delta_i) = \max\left(1, \left\lceil \frac{|\Delta_i|}{(1+s)\Delta_{\min}} \right\rceil\right)$$

URE satisfies Differential privacy: Two claims are proven in the paper:

Splitting is differentially private
SURE is differentially private, i.e. supposing we have a sequence of admissible trades $\Delta$ such that the height of the random search tree is $O(\log n)$ and all trade sizes are unique, then randomly permuting the trades can be made into a $(\mu \log n, \delta)$-differentially private algorithm.

Ultimately, this helps us show that URE is $(\epsilon, \delta)$ differentially private with

$$\epsilon = \mu \log n + \max_i \Delta_i$$$$\delta = F^{-1}\left(O\left(\frac{1}{\epsilon}\right)\right)$$

These results suggest that permuting and splitting up trades is a simple and viable mechanism for adding differential privacy to CFMMs.

One thing to note is we can achieve this differential privacy on any chain today, which has a verifiable random function because this can help us create the noise to add and also help us randomly permute the orderings of the trades. This is because, as we saw in our example of differential privacy originally, we need a randomness source to obscure the data, and doing this in a trustless way is only possible by Verifiable random functions or VDFs (Verifiable Delay Functions) that serve as randomness beacons.

Deriving from the works of “Differential privacy in CFMMs” and “Improved Price oracles: CFMMs”, the following things can be observed:

In the Differential privacy paper, the authors show the trade-off between the cost of privacy and the level of privacy. When a trade is split into two smaller trades combined with shuffling the order of transactions, this leads to better privacy for users while worsening their price impact.
In the CFMM paper, It is demonstrated that splitting trades and executing them on a CFMM leads to worse utility, i.e. worse price execution (the user pays a higher price for the same quantity of an asset).

These results let us conclude that CFMMs trade off privacy for utility.

Understanding Credible Optimal Auctions via Blockchains

Mon, 03 Apr 2023 19:59:04 +0000

Exploring and understanding the paper “Credible, Optimal Auctions Via Blockchains” by Tarun Chitra et al.

Auctions everywhere

Google Ad Auctions

Every day we find ourselves on the internet for various reasons, be it searching for things, buying, selling and surfing/browsing the internet for work or otherwise. The economy of the internet runs on ads. Every time you visit a website, you are welcome with various ads by companies; website publishing companies sell these “ad slots” in the website to advertisers. Website publishers sell about 5 trillion ads to advertisers yearly, i.e. 13 billion ads daily, just in the US. These ads generate about 20 billion$ worth of revenue annually for companies, which is again just in the united states.

How does one go about selling an advertisement to an interested party? There is an exchange where website publishers list the ad space for sale, and advertisers buy it from them. The Department of Justice recently accused Google of justice for “monopolizing Digital Advertising technologies”.

A few of the complaints against google

Ad spaces are sold using auctions, where google acts as the auctioneer, the publishers are the sellers and businesses interested are bidders. Over the past decade, it has been found that google engaged in various manipulation practices to ensure its monopoly in the space and extract as much value as possible in the system. These include:

They configured google ads to bid higher to increase the cost of advertising to the detriment of advertising customers/benefit of publishers. This led to publishers choosing google over competitors and google extracting value.
In second-bid auctions, google usually placed two bids to anchor the auction price higher so that they could profit from the ultimate settlement price of the auction.
When google did move to the first price auctions, it justified the move to prevent bidders from preferencing rival ad exchanges.

While a thorough coverage of the antitrust case is not this article’s purpose, Auctions are ubiquitous in the shadows of everything we use on the internet. Participants in auctions need to trust the auctioneer not to engage in unfair practices, thus undermining the experience and the real value of the assets auctioned for the sake of profits.

Art Auctions

Another class of assets quite exclusively exchanged/sold/bought through auctions is Art. An auction house’s private sales data aren’t available to the public.

There’s been a recent trend of the cost of expensive Art rising. In 2018, Sotheby’s, Christie’s and Phillips sold art pieces worth 1 million dollars or more, cumulatively yielding 7.44B $ in volume, which has grown to 8.15B $ by 2022. The average art piece price in this bracket was 5.3M $ in 2018, while it was 6M $ in 2022.

The Sotheby’s art auction is a first price ascending bid auction where bidders continuously bid till the highest bid wins. Also, a reserve price ensures that the seller makes some profit relative to how they value the art piece. Many bidders are not present in the auction room at all but instead bid over the internet or telephone. Christie’s and Sotheby’s are legally permitted to call out fake bids to give the impression of higher demand.

The recent work of Tarun Chitra et al. in the paper “Credible, Optimal Auctions Via Blockchains” focuses on how we can achieve auctions which hold the auctioneer accountable while also being optimal. Smart contracts allow one to extend the concept of credibility to settings where the auctioneer does not have a reputation/it also enables you to hold trustless auctions if you have an asset you want to sell. As digital frontiers expand, a significant need for trustless credible auctions also grows.

A brief primer on Auctions

Auctions are a mechanism for buying and selling goods and services in marketplaces where goods are sold. From art auctions to online marketplaces, auctions allocate resources, determine prices, and facilitate transactions between buyers and sellers. However, not all auctions are created equal, and the design of the auction can have a significant impact on its outcome.

There are several types of auctions seen in the wild, but the major ones are as follows:

English auction: Also known as an “open ascending price auction”, this is the most common type of auction where participants openly bid against each other, with each bid higher than the previous one. The auction ends when no one is willing to bid higher, and the highest bidder wins. We usually see this type of auction on Foundation for Art 1/1s.
Dutch auction: Also referred to as an “open descending price auction”, the auctioneer starts with a high asking price and gradually lowers it until a bidder accepts the current price or the reserve price is reached. We saw a form of this in Gradual dutch auctions in the recent art gobblers mint.
First-price sealed-bid auction: In this type of auction, participants submit their bids in sealed envelopes without knowing the bids of others. Once the bids are opened, the highest bidder wins the item and pays the price they bid. These are also known as first-price auctions.
Second-price sealed-bid auction: Also known as a Vickrey auction, participants submit sealed bids, and the highest bidder wins. However, the winner pays the price offered by the second-highest bidder instead of their bid. These are also known as second-price auctions.

While the 3rd and 4th auctions are sealed bids as in private auctions where other participants are not aware of the bids of other participants, these aren’t possible on-chain today.

Credible Auctions: A Trillema

The paper “Credible Auctions: A trilemma” by Akbarpour et al. primarily studies incentive compatibility for an Auctioneer.

As discussed earlier, Second price auctions are a type of auction in which the highest bidder wins the auction but pays the second-highest bid as the price rather than their bid. In a second price auction, bidders are incentivised to bid their true valuation of the goods, as they will never pay more than their valuation and may pay less if other bidders bid less than their true valuation. The issue with a Second-price auction is that the auctioneer can profit by exaggerating the second-highest bid. Therefore special arrangements have to be made to tie the auctioneer’s hands.

If the way of communicating bids is public, then the problem is trivial since everyone is aware of everyone else’s bids, so it’s trivial and strategy-proof. Most real-world auctions are not publicly communicated. If we then assume the auctioneer communicates privately with each bidder, this allows the auctioneer to misrepresent any bidders’ preferences to other bidders.

Some properties of auctions

Suppose there’s a protocol of strategy profiles for bidders and also an extensive-form mechanism. This mechanism decides how the auctioneer runs the auction. Starting from initial history, the auctioneer conveys a message to the bidder, who then bids. The auctioneer keeps collecting the bids until they reach a terminal history, and the auctioneer chooses who wins.

Assume that there’s some utility function for the auctioneer. This could be a variety of things, but usually profit. By participating in a protocol, each bidder observes the communication between himself and the auctioneer and some features of the outcome. Even if the auction deviates from the assigned strategy of the bidder, the observation could have some innocent explanation. For example, in a second price auction, when a bidder bids 100$ and wins and has to pay 99$, this observation has an innocent explanation, the second highest bid was 99$.

Given any protocol, some deviations may be safe, i.e. for every bidder’s observation, there is some innocent explanation. Thus in a second price auction, the auctioneer can safely deviate by exaggerating the second-highest bid. Instead of exaggerating, the auctioneer can also communicate differently. E.g., the seller chooses a price, and the auctioneer tells it to the buyer and the object is sold to the buyer only if they accept (no negotiations) and the auctioneer takes a commission. The safe deviation here is the auctioneer can quote a higher price to the buyer and pocket the difference. A protocol is “credible” if running the mechanism is incentive compatible for the auctioneer over any safe deviation, i.e. the auction prefers playing by the book.

A first price auction is “static” - each bidder is called to play exactly once and has no information about the play history when selecting his action/bid.

The ascending auction is “strategy-proof”. Thus it needs no or very less strategic planning from bidders.

The second price auction is “static” and “strategic-proof”. It combines the virtues of the first price auction and the ascending auction but is not credible, as seen in our examples.

Ultimately the paper presents the auction mechanism trilemma. Static, strategy-proof or credible. An optimal auction can have any two of these properties but not all three at once. Should an auction be static, strategy-proof and credible? While an opportunity for a nuanced discussion presents itself, here are some base arguments for every parameter. Advertisement auctions must be conducted in milliseconds or faster, so latency disallows multi-round auctions, so static auctions might be preferred. Strategy-proof-ness matters when bidders are inexperienced or have opportunities for rent-seeking. Credibility matters, especially when bidders are anonymous to each other or require that the bids are kept private. In this case, an auctioneer has all the information to deviate safely.

Ultimately, The paper shows that the ascending price auction (with reserves) is credible, and under some conditions, it is a unique, credible strategy-proof optimal auction. The conditions being it requires an unbounded number of rounds.

Deferred Revelation Auctions

In the paper, “Credible, Truthful and Two-Round (Optimal) Auctions via Cryptographic Commitments”, Ferreira et al. design an auction that avoids the trilemma and provides a truthful, revenue-maximizing, credible, two-round auction under the assumption of basic cryptographic primitives.

The cryptographic primitive that they use is commitment schemes. A commitment scheme is a cryptographic primitive that allows a person to commit to a chosen value while keeping it hidden from others. It ensures that the person cannot change the value after committing to it, which can be revealed later. Commitment schemes are essential for maintaining trust, integrity, and security in various cryptographic applications.

So the skeleton of the auction is as follows:

Ask each bidder to commit to their bid
forward these commitments to all other bidders
ask each bidder to reveal
forward the revealed bids to all other bidders

This auction is truthful, revenue optimal and two-round. The auctioneer can deviate primarily by submitting fake bids, but if we assume that all committed bids must be revealed, then the auctioneer cannot deviate and submit fake bids. While this is sound, auction design should involve the edge case where some bids are concealed, and the auction is not stalled. For this, the authors suggest fining any bidder that does not reveal and paying this fine to the winning bidder. The auctioneer now faces a tradeoff between submitting as many fake bids as they want and paying the penalty for every bid they conceal.

Ferreira et al. paper designed Deferred revelation auctions as a communication efficient auction assuming the existence of cryptographic commitments and that all bidder valuations are MHR (Monotonic hazard rate). They also showed that DRA is not credible in settings where bidder valuations are α-strongly regular unless $\alpha > 1$. We will take a brief moment to understand MHR and α-strongly regular distributions.

α-strongly Regular Distribution

The paper “The sample complexity of Revenue Maximization” by Cole and Roughgarden originally studied alpha-strongly regular distributions. The main focus of their work was to study auctions, particularly in Myerson’s auction, when distributions are known only approximately and how to analyze the resulting expected revenue as a function of the number of samples.

where the virtual value function is given as follows

Alpha-strongly regular distributions, also known as alpha-regular distributions, are a class of probability distributions used in auction theory to analyze bidder valuations. An alpha-strongly regular distribution satisfies the following condition for some constant alpha (α):

The virtual value function is non-decreasing.

The virtual valuation function is derived from the probability distribution of bidder valuations and is crucial in determining the optimal auction mechanism. In particular, the α-strongly regular condition guarantees that the virtual valuation function is well-behaved and allows for tractable analysis of the auction.

Alpha-strongly regular distributions are a more general class than Monotone Hazard Rate (MHR) distributions. MHR distributions are a special case of α-strongly regular distributions, where $\alpha = 1$. This means that all MHR distributions are α-strongly regular, but not all α-strongly regular distributions are MHR.

What does “bidders valuations” being MHR mean? It means that the valuations satisfy the Monotone Hazard Rate (MHR) condition. The Monotone Hazard Rate condition states that the ratio of the probability density function to the complementary cumulative distribution function is non-decreasing. In simpler terms, the probability of a higher bid occurring increases as the bid value increases. MHR distributions have certain desirable properties in auction design, such as leading to higher revenue for the seller and promoting more efficient allocations. Intuitively, MHR bidder valuations can be thought of as having exponential tails where the buyer valuation/bidding follows an exponential curve.

Credible auctions on blockchain

Summing up what we’ve seen so far:

An auction is credible if safe deviations cannot increase the auctioneer’s revenue
Akbarpour et al. showed the auction trilemma, and that credibility cannot coexist with truthfulness and bounded communication complexity.
Ferreira et al., in private channel auctions, cryptographic auctions (specifically cryptographic commitments) can achieve credibility, truthfulness and bounded communication complexity if buyer valuations satisfy a regularity condition. The collateral requirements ensure that an auctioneer cannot add and refuse to reveal fake bids for free (since there’s a penalty).

Can we design auctions without requiring regularity conditions? Ferreira et al proposes the Ascending Deferred Revelation Auction (ADRA). It is similar to an ascending auction, but prices increase exponentially from one round to the next. Even this auction, if done using commitment schemes, can ensure that the auction is strategy-proof and revenue optimal, but it has some limitations. One crucial underlying assumption that the Ferreira et al. paper has is that the buyers and auctioneers communicate over private channels. In other words, All communication between bidders can only happen through the auctioneer. This needs the auctioneer, to be honest. Otherwise, the auctioneer can launch a man-in-the-middle attack by censoring and injecting messages.

What if there was a censorship-resistant public channel over which the commitments could take place? In this case, any message sent by a bidder will be seen by all other bidders and any message received by a bidder is also received by all other bidders. In Chitra et al., the authors ask, “How would auctions look like if they were conducted on a blockchain?” and “what guarantees or features would this auction have?”. They find the following.

They expand the work done by Ferreira et al. and show that Deferred Revelation auctions (DRAs), when done over a secure, censorship-resistant ledger, are credible for α-strongly distributions as long as $\alpha > 0$. In simpler words, conducting auctions over public channels can expand the domains for which these options are credible.
DRAs are not credible if buyer valuations are regular or α-strongly regular for $\alpha = 0$, i.e. blockchains are not a panacea and don’t make auctions universally credible.

The paper designs a single-item auction with $n$ buyer auctions with independent valuations and shows that it is credible. Blockchains in the past decade have then and again proven to be the gold standard of censorship-resistant public channels where agents can not only communicate but also do computations. This also makes it very appealing for mechanism designers because “a public channel” in its true sense does not exist in the credibility framework of Akbarpour et al. because they assume auctions would run over the internet, which is private, i.e. happen across private channels.

An intuitive way of thinking about auctions, and this paper is:

Usually, in single-item auctions, you have multiple bidders, let’s say who bid 10$, 5$, 20$ etc to win the item, ultimately the winner in case of a second-price auction would be the person who bid 20$ but would end up paying only 10$.
There are some properties that we want the auctions we design to have, the most important of them being the auction should be credible,
- credibility in auctions can be thought of as when the auctioneer who wants to profit from the auction should conduct it fairly and not resort to malicious behaviour like misquoting the bids to increase his profit share.
While it was earlier shown that the properties we like are impossible to attain in an auction design, Deferred revelation auctions were proposed by Ferreira et al. who found a 2 round auction design with commitment schemes to make it happen.
In formulating auctions, we need a way to model bidder valuations since we cannot assume the values of the bids directly. This is because it would not generalise.

We model bidder valuations by assuming they are sampled as values from a distribution, i.e. the user’s bidding behaviour could follow values along distributions.
DRA, aka Deferred revelation auctions, showed that the auction design is reliable when you assume that the bidder valuations were Exponential distributions.
- Exponential distributions are used infrequently and only represent a subset of all possible distributions. (D2 in the image above)
The paper “Credible auctions on Blockchains” expands the distributions we can assume bidders to have, which is a far larger class of distributions compared to just exponential distributions. For example, this includes Pareto distributions. (D1, D3 etc., in the image above)
While the paper ultimately mentions that when $\alpha = 0$, the auction is unreliable.
A major takeaway of the paper (which might not be understood) is distributions which are $0 < \alpha < 1$ are a far bigger class of distributions than $\alpha > 1$, which was proved in the earlier paper by Ferreira et al. and which encompasses just exponential distributions.

Where could these credible auctions be used?

NFTs: NFT volumes have reduced in the past six months but have seen a steady influx of participants and transactions overall. The authors of the paper “A framework for single-item NFT auction mechanism design” show that NFT auctions cannot achieve incentive compatibility and collusion resistance. DRAs can be one answer to this, especially for auctions for 1/1 art pieces like those on foundation.

Foundation NFT volumes

Opensea Monthly volume

MEV (Miner extractable value): Flashbots runs the most popular auction on ethereum, and the majority of MEV has been extracted through these auctions run by validators. What does an auction look like?
- MEV refers to the excess value validators can extract by reordering, adding or removing transactions to a blockchain.
- Bidders (also known as MEV searchers) bid on transaction priority for sequences of transactions.
- If a bidder wins an auction, they are guaranteed their bundle of transactions is included in the block, and the auction revenue is distributed to the validator who proposes this block.
- Flashbots currently runs these first-price, sealed bid auctions as a monopoly, but there is an active need for decentralizing this aspect of auctions. The condition stems from the ability to censor or delay time-sensitive transactions as these preferences can be expressed by the auctioneer, in this case, Flashbots.
- A credible and truthful auction can make sure nobody can manipulate the outcomes.

In the current transaction market, the block proposer (today: a miner, post-merge: a validator) directly chooses which transactions to include in the next block by looking at which transactions in the mempool pay the highest priority fee.

Proposer/builder separation (PBS) fixes this by splitting the block construction role from the block proposal role. A separate class of actors called builders build exec block bodies (essentially an ordered list of transactions that becomes the main “payload” of the block), and submit bids. The proposer’s job is only to accept the exec block body with the highest bid. Notably, the proposer (and everyone else) does not learn the contents of any exec block body until after they select the header (and hence the body) that wins the auction.

From “Notes of Ethereum: Increasing Censorship Resistance through PBS” by Vitalik Buterin

Proposer-Builder separation: In proof of work consensus, a block has to be determined before validator selection, while in proof of stake, the block content can be determined after the validator is selected. Thus the validator can auction away the privilege to build the next block. These are single-item (just one block) auctions. This separation between the proposer and the builder is meant to increase the decentralization of block production in blockchains.
- The collateral requirements of DRA can be used to formalize censorship-resistant bounds, and perhaps even under auctioneer deviation, a PBS auction following a DRA can make it incentive compatible.

DRA over a ledger

The overall protocol looks similar to what was done in a normal DRA, where the auctioneer has a private channel for each bidder, except for some details.

Commitment Phase

The auctioneer writes into the ledger, and each buyer draws a random number from their distribution and commits to the ledger. Each buyer either aborts or deposits collateral, which all participants observe.

Revelation Phase

Each buyer reveals their commitment, and all other buyers observe this.

Smart Contract Resolution

The highest bidder is awarded the item, and the smart contract transfers the collateral from those who didn’t reveal their commits to losing the collateral. Those who didn’t win and revealed their commits are given their collateral back.

Tiebreaking

All ties are broken lexicographically.

The significant difference between this mechanism and Ferreira et al. is the public communication channel ensures all buyers receive messages. Buyers do not rely on the auctioneer’s reputation for receiving the item or their collateral back because all of this is implanted through a smart contract.

The paper’s proofs are beyond this blog’s scope but are straightforward and concisely written. The paper can be found here. One thing to note is that commitments can be done using ZKPs or alternative commitment schemes like EIP-5732 when it is added.

The paper ultimately shows that DRA is credible for all α-strongly regular distributions for $\alpha > 0$ and not for regular distributions, i.e. where $\alpha = 0$. This leaves an open question and design problem for other settings, including regular distributions and multi-item auctions.

References

Credible Auctions: A Trilemma — Akbarpour and Li; the original trilemma paper.
Credible, Truthful and Two-round (Optimal) Auctions via Cryptographic Commitments — Ferreira and Weinberg; the paper introducing Deferred Revelation auctions.
The Sample Complexity of Revenue Maximization — Cole and Roughgarden; the paper introducing α-strongly regular distributions.
Applications of α-strongly regular distributions to Bayesian auctions — Cole and Rao; the paper expanding the work on α-strongly regular distributions.

To listen to a talk on this paper, you can look at

From auctions.google.com to auctions.best, Tarun Chitra (Gauntlet) | MEV [re]search-athon - YouTube youtube.com

Understanding "Towards a Theory of MEV I"

Sun, 11 Dec 2022 16:05:47 +0000

Originally introduced in the paper, Flashboys 2.0, Maximal(or Miner) Extractable value is the value that is captured by miners or validators from users in a network. While exotic forms of MEV exist, the most common form of MEV comes from reordering user transactions to maximize fees or frontrunning users while making a trade.

This blog is a primer/explainer for the paper, “Towards a Theory of Maximal Extractable Value I: Constant Function Market Makers”.

The Anatomy of a transaction

While the granular picture of a transaction from genesis to getting included in a block is perhaps a much longer article, We can take a simple look at it before we understand the nuances of how MEV works practically.

The users submit transactions which appear in the mempool, and there are various views (one way to think of a view is a permutation of the ordering or submission of transactions). MEV searchers can have a different view or design one to create a bundle (with some metric to optimize, almost always for profit). Sometimes the user could directly submit the transaction to the searcher and this is also termed “Order flow” in common parlance. The view/bundle is submitted to the validator for inclusion and this accepted bundle/a bunch of accepted bundles together create the block.

The frontrunning attack in practice is quite simple, after seeing a users trade for buy an asset X, a transaction is submitted before for X which pushes up the cost in case of buying after which the user’s transaction gets executed at this updated value which results in worse price execution and another transaction is submitted which sells the asset, the difference in this frontrunning i.e buying before a transaction is submitted and selling after is usually the profit (deducting gas costs or bribes to the miner/validator for allowing these transactions). This is also known as a Sandwiching attack and is mostly done on Automated market makers. While multi-block MEV might exist or does exist in some forms, most MEV today is extracted in single blocks.

The paper tries to answer two critical questions about MEV that occurs in Constant Function market makers (DEXes that we use belong to this broader generalization of the concept):

In a Single CFMM with two assets, how much does reordering user trades affect the excess price impact caused by sandwich impacts?
In the case of a network of CFMMs trading multiple assets, how much does the presence of sandwich attackers on the network affect the routing of trades?

The paper overall, the first in many, analyzes game theoretic properties of MEV ultimately establishing the following:

In the case of reordering MEV, the maximum price impact caused by reordering of sandwich attacks in the sequence of trades is on the order of $O(\log n)$ where $n$ is the number of user trades.
In the case of routing MEV, the existence of MEV both degrades and counterintuitively improves the quality of routing.
The paper also shows that if the impact of a sandwich attack is localized, then the price of anarchy is constant. [Similar to the tragedy of commons which talks about selfish behaviour by people in public goods systems, the price of anarchy is a game theoretic formalism that measures how much the efficiency of a system degrades by the selfish behaviour of its agents/users]. In simpler words, sandwich attacks do not greatly impact routing quality in networks of CFMMs.
The paper analyzes MEV through a theoretical lens and discusses the economic properties of systems with MEV. The focus of this paper is mostly on user-made transactions with a particular type of Automated market maker i.e. Constant function market makers.

One of the fundamental reasons for being able to extract MEV by a miner is their ability to see all transactions publicly before committing to validate a block. In the paper, “Differential privacy in CFMMs”, Angeris et al connect MEV profits in AMMs with a loss of user privacy. In studies that were done in differential and federated machine learning, it was found that a model’s utility to its users generally goes down as user privacy is increased. This can be attributed to the fact that most privatization of data includes adding noise to inputs and models trained on this data suffer from data quality loss. The maximum loss to a user from lack of privacy is also a major factor in the calculation of maximum profit through MEV. This paper also studies a tradeoff between privacy and utility for users of CFMMs.

Deriving from the works of “Differential privacy in CFMMs” and “Improved Price oracles: CFMMs”, the following things can be observed:

In the Differential privacy paper, the authors show the trade-off between the cost of privacy and the level of privacy. When a trade is split into two smaller trades combined with shuffling the order of transactions this leads to better privacy for users while worsening their price impact.
In the CFMM paper, It is demonstrated that splitting trades and executing them on a CFMM leads to worse utility i.e. worse price execution (the user pays a higher price for the same quantity of an asset).

These results let us conclude that CFMMs trade off privacy for utility.

Sandwich Attacks

As mentioned before, Frontrunning trades involve placing an order before a user’s order which results in a worse execution and then selling to book a small profit. These trades are also called Sandwich Attacks.

Usually in Centralized exchanges when users place an order they mention a price at which they want their order to be executed and these are called limit orders. While in highly volatile environments it might not be possible to fill the entire order at a price which results in slippage.

Decentralized exchanges don’t have “limit orders” and the limit price is expressed in the form of a slippage limit. The way to think about it is, If I were buying a token (X at 100) and I express my slippage limit as 0.5%, the worst price I’m willing to settle at is 100.5$. While 100.5$ is not terrible, Decentralized exchanges have worse slippage compared to Limit order books (that run on centralized exchanges), more often than not slippage limits are around 2-5% depending on how much liquidity is available for a token. An easy notion to build is low liquidity markets have more volatility and in turn more slippage for the same size of order when compared to a highly liquid market. Why is this relevant?

Grug 🪨 @grugcapital

When you set your slippage to 5% you are expressing the highest price you’re willing to pay for a given trade. It would be irresponsible of me not to meet you there!

Nov 19, 2022

When an adversary (miner/MEV searcher) looks at a user’s trade with the slippage order, they can now submit a trade which pushes the price of the token to the worst possible execution price for the user. If a user wishes (X at 100) and places a slippage of 5%, the worst possible price the user would settle for is 105$, an adversary seeing this can buy the token $X pushing the price to the highest slippage limit at which the user buys the token and the adversary sells his token back thus booking a profit.

Uniswap

Before establishing Sandwiching for generic CFMMs, we establish it for Uniswap.

In the paper, “An Analysis of Uniswap Markets”, Angeris et al established that a trade in Uniswap is valid if

$$(R_\alpha - \Delta_\alpha)(R_\beta + \gamma\Delta_\beta) = k.$$

Also written as,

$$(R_A - \Delta')(R_B + \gamma\Delta) = R_A R_B$$

where the equation is an expression for a pool between two tokens $A$, $B$, $R_a$ is the reserves of token $A$, and $R_b$ is the reserves of the token, $\Delta$ is the user trade size of token $B$ and $\Delta'$ is the amount of $A$ tokens he gets, and $1 - \gamma$ is the fee parameter. A quick note that it’s called a constant product market maker because if the fee parameter is zero i.e. if $\gamma$ is 1 then the product of reserves should always result in $R_a R_b$ i.e. it should be constant.

The price of $A$ in terms of $B$ is simply given by $R_a / R_b$, while this is the price quote by Uniswap, for a trade size of $\Delta$ it slightly changes to (this is also called the marginal price of trade $\Delta$)

$$p_{AB}(\Delta, R_A, R_B) = \frac{R_A - \Delta'}{R_B + \gamma\Delta}.$$

Ultimately, the amount of output token a user receives using the marginal price is

$$G(\Delta) = \frac{1}{\gamma}\left(\frac{-R_A R_B}{R_A + \Delta} + R_B\right)$$

Now we would like to incorporate slippage into this, As described earlier the slippage limit $\eta$ can be between 0 and 1. It is the amount of the price impact a user is willing to tolerate. A slippage limit is essentially important because, in the case of multiple trades being submitted at a price, it might so happen that there is some price impact for a few trades. Taking the example of two trades $\Delta_1$ and $\Delta_2$, a miner might execute one trade before the other so they might not get the exact price that was displayed to them, but by expressing a slippage limit a user declares that they are not ready/will not accept less than $(1 - \eta)$ times the amount.

The minor cannot execute a trade $\Delta_1$ after $\Delta_2$ unless

$$G(\Delta_1 + \Delta_2) - G(\Delta_2) \geq (1 - \eta)G(\Delta_1)$$

where $\eta$ is the slippage limit.

When a slippage limit is too large, the above equation becomes a strict inequality and allows for an adversary/attacker i.e. MEV searcher to construct a trade $\Delta^{\text{sand}}$ which

$$G(\Delta + \Delta^{\text{sand}}) - G(\Delta^{\text{sand}}) = (1 + \eta)G(\Delta)$$

So essentially our “attacker” fills the slack in the equality constraint, worsens execution for the user and books a profit. An easy intuitive way of thinking about it is the attacker is taking advantage of the margin. The trades $(\Delta^{\text{sand}}, (\Delta, \eta), \Delta^{\text{sand}'})$ constitute a sandwich attack i.e. the order before the user order to pump the price, the user order executed with his slippage limit and the subsequent sale of the token by the sandwich attacker to book the profit.

Quickly going over the case for Uniswap,

Note we only take the positive root when solving the quadratic equation because it also satisfies an intuitive idea that when the slippage is 0 i.e. $\eta = 0$, $\Delta^{\text{sand}} = 0$.

Solving the equation for $\Delta^{\text{sand}}$ and plugging in other values actually give you the exact quantity of $\Delta^{\text{sand}}$ that you can extract.

Constant Function Market Makers

While Uniswap is a part of CFMMs where the function is a constant product, we would like to generalize these results to all kinds of functions in CFMMs.

Recall from Improved Price Oracles: Constant Function Market Makers, the definition of a trading function (an explainer can be found below).

Building Blocks of Primitive - Constant Function Market Makers [CFMMs] 0xemperor.net

Similar to how we defined it for Uniswap, A CFMM contract only accepts a trade if,

$$\psi(R, R', \Delta, \Delta') = \psi(R, R', 0, 0),$$

Where $\psi$ is the trading function whose domain is from: $\mathbb{R}^2 \times \mathbb{R}^2 \to \mathbb{R}$.

The marginal price for a trade size of $\Delta$ for a CFMM is:

$$g(\Delta) = \frac{\partial_3 \psi(R, R', \Delta, \Delta')}{\partial_4 \psi(R, R', \Delta, \Delta')}.$$

here $\partial_3$ and $\partial_4$ are the partial derivatives wrt to the ith argument. $g$ is known as the price impact function as it represents the final price (marginal price) of a trade.

There are two CFMM properties that we care about $\alpha$-stable and $\beta$-liquid, they are satisfied if

A CFMM is $\alpha$-stable if

$$g(0) - g(-\Delta) \leq \alpha\Delta$$

A CFMM is $\beta$-liquid if

$$g(0) - g(-\Delta) \geq \beta\Delta$$

Basically, $\alpha$-stable means whenever a non-negative trade size of $\Delta$ does not change the market price by more than $\alpha\Delta$, it places a linear upper bound on the maximum price impact that a bounded trade can have. Similarly, $\beta$-liquid means that there is some price slippage when a token is sold but is linearly bounded from below by at least $\beta\Delta$.

One important property of $g$ i.e. the price impact function is the ability to calculate

$$\Delta' = \int_0^{-\Delta} g(t)\,dt$$

where $\Delta'$ is also can be defined as $G(\Delta)$ which is the forward exchange function, this is the amount of output token received for an input size of $\Delta$.

We can also define two-sided bounds for $g(\Delta) - g(0)$, as follows

Exactly similar to the Uniswap case, we found that when the slippage is set too high, then $\Delta^{\text{sand}}$ satisfies, this holds generally true for CFMMs as well:

$$G(\Delta + \Delta^{\text{sand}}) - G(\Delta^{\text{sand}}) = (1 - \eta)G(\Delta)$$

We derived a case for the exact $\Delta^{\text{sand}}$ for the Uniswap case, we can also find the $\Delta^{\text{sand}'}$ i.e. the amount of token that the miner needs to sell to stay risk neutral by solving

$$\Delta^{\text{sand}'} = \Delta^{\text{sand}} + \Delta - G^{-1}(G(\Delta + \Delta^{\text{sand}}) - G(\Delta^{\text{sand}}))$$

Where $G(\Delta)$, as we noted earlier, is the forward exchange function i.e. the amount of output token received for an input size of $\Delta$. Both $\Delta^{\text{sand}}$ and $\Delta^{\text{sand}'}$ are in units of input token, and finally, we arrive at an equation to calculate the profit:

$$\begin{aligned} \mathrm{PNL}(\Delta, \eta) &= \Delta^{\text{sand}'}(\Delta, \eta) - \Delta^{\text{sand}}(\Delta, \eta) \\ &= \Delta - G^{-1}(G(\Delta + \Delta^{\text{sand}}) - G(\Delta^{\text{sand}})) \end{aligned}$$

The PNL equation here is for net profit and loss.

A note for a study: It would be interesting to do a study of :

1] What potential profits could have been possible if every sandwich attack in the past few years had been sandwich-optimal? Of course, these calculations have to include fees which are missing from our equation.

2] What is the general way sandwich bots calculate token size to be sold or bought does it vary or is it adjacent to the equations provided above?

Bounds on Sandwich attack profitability

To bound sandwich attacks, we first need to reason about the expected size of a sandwich attack $\Delta^{\text{sand}}_i$ given a sequence of trades $\Delta_1, \Delta_2, \ldots, \Delta_n$.

While a complete treatment of all the results might not be possible, we will quickly go over a few definitions and subsequent results from the paper.

A function $G(\Delta)$ is $(\mu, \kappa)$ smooth if there exists $M > 0$, such that $\Delta \in [0, M]$ then there exists $\mu, \kappa > 0$ such that

$$\kappa\Delta \leq G(\Delta) - G(0) \leq \mu\Delta$$

$\mu, \kappa$ are different from the stability and liquidity constants we described for the function $g$ earlier.

This definition helps us claim,

$$\text{If } \eta \geq 1 - \frac{\kappa}{\mu} \text{ then we have } \Delta^{\text{sand}}(\eta, \Delta) = O(\eta)\Delta$$

This bound simply says that the size of a sandwich attack is linear in the slippage limit. The slippage limit has to be larger than the curvature ratio.

In addition to $G(\Delta)$ being $(\mu, \kappa)$ smooth if $g(\Delta)$ is $\beta$-liquid, then there exists $\gamma = 1 + \Theta(\sqrt{1 + \eta})$ such that

$$\Delta^{\text{sand}} \geq \left(\frac{\mu}{\beta} - \Delta\right)\gamma$$

This also helps us analyze constant sum market makers, in “Constant Function Market Makers: Multi-Asset Trades via Convex Optimization”, Angeris et al show that $\beta = 0$, therefore plugging into this equation we find that since there is no price impact when one executes a sequence of trades, there is no sandwich profit possible in constant sum market makers.

Ultimately, this series of claims allows us to bound the total trade size and the profit from a sandwich attack by

Simply put, given liquidity and slippage conditions are met, profit and price impact are linear in $\gamma$.

Reordering MEV

In the case of reordering MEV for sandwich attacks, we assume the following

A block size of $h$ that can process $n$ trades
A sequence of trades $T_n = \{(\Delta_1, \eta_1), (\Delta_2, \eta_2), \ldots, (\Delta_n, \eta_n)\}$ where $3n < h$ so that there are enough slots to sandwich attack the trades.
$\eta_1, \eta_2, \ldots, \eta_n$ are the slippage limits for the trades which are lower bounded by a single $\eta$

We ultimately seek to understand the quantity called as the cost of feudalism, given by

$$\mathrm{CoF}(T_n) = \frac{\mathbf{E}_{\pi \sim S_n}\left[\max_{i \in [n]} |\mathrm{PNL}_{\pi(i)}(T_n) - \mathrm{PNL}_i(T_n)|\right]}{\mathbf{E}_{\pi \sim S_n}\left[\frac{1}{n}\sum_{i=1}^{n} |\mathrm{PNL}_{\pi(i)}(T_n) - \mathrm{PNL}_i(T_n)|\right]}$$

A small digression into what feudalism is and how to intuitively think about this, feudalism was originally a system in which people were given land and protection by people of higher rank, and worked and fought for them in return. In the context of blockchains, we might think of miners offering tx inclusion in the block as the service while the cost is paid in the form of MEV.

We are establishing the cost of feudalism as the ratio that compares the profit captured from sandwiching for the worst affected user to the average user. It characterizes the maximum amount that any individual user’s price execution might be affected by reorderings. We want to upper bound the numerator so we know what the worst case can be and lower bound the numerator so we know what is a cost that at least needs to be paid.

Trades are roughly mean reverting.
$\mathrm{CoF}(T_n)$ can be bounded as a function of the curvature constants $\mu, \kappa, \alpha, \beta$, the slippage limit $\eta$ and the trade drifts $u_i$.
- $u_i$ is defined as the partial trade drift for the ith order, which is defined as

$$\tilde{u}_i = \sum_{j=1}^{i} \xi_j.$$

where $\xi_j$ is defined as

$$\xi_j = \Delta^{\text{sand}}_j + \Delta_j - \Delta^{\text{sand}'}_j.$$

and we know that the $\Delta^{\text{sand}}_i$ and $\Delta^{\text{sand}'}_i$ terms satisfy (from the previous section)

$$\begin{aligned} G\left(\Delta^{\text{sand}}_i + \Delta_i + \sum_{j=1}^{i-1} \xi_j\right) - G\left(\Delta^{\text{sand}}_i + \sum_{j=1}^{i-1} \xi_j\right) &= (1 - \eta)G(\Delta_i) \\ \Delta^{\text{sand}'}_i &= \Delta^{\text{sand}'}_i + \Delta - G^{-1}(G(\Delta^{\text{sand}}_i + \Delta_i) - G(\Delta^{\text{sand}}_i)) \end{aligned}$$

If a set of trades is strongly local, i.e. sandwich trade locality means that it is never more profitable to sandwich bundles of transactions versus sandwiching them individually, then the following hold:

$$q(\mu, \kappa, \alpha, \beta, \eta, \tilde{u}_i)\Delta_i \leq \mathrm{PNL}_i \leq p(\mu, \kappa, \alpha, \beta, \eta, \tilde{u}_i)\Delta_i$$$$\mathop{\mathbf{E}}_{\pi \sim S_n}\left[\max_{i \in [n]} |\mathrm{PNL}_{\pi(i)} - \mathrm{PNL}_i|\right] = O(\log n)$$$$\mathop{\mathbf{E}}_{\pi \sim S_n}\left[\frac{1}{n}\sum_{i=1}^{n} |\mathrm{PNL}_{\pi(i)} - \mathrm{PNL}_i|\right] = \Omega(1)$$

Basically says that the maximum profit captured from sandwiching the worst affected user is $O(\log n)$ while the average affected user’s PNL is constant (?).

Plugging these results back into our equation for the Cost of feudalism we get that,

The cost of feudalism is $O(\log n)$ i.e. the cost of feudalism just goes up linearly even if the number of trades goes up exponentially.

Tarun Chitra @tarunchitra

Replying to @tarunchitra

Using these bounds, we show the first surprising result: the worst-case sandwich profit is often only O(log n) times worse than the average-case sandwich — this suggests that the compounding of sandwich attacks is relatively limited (even if the whole block is sandwich attacks)

Jul 18, 2022

Routing MEV

Routing MEV for sandwich attacks intuitively means the value that a miner/sandwich attacker can extract from a user’s trade over a network of CFMMs. In the paper, Optimal Routing for Constant Function Market Makers, Angeris et al demonstrate that in a universe of $n$ tokens where also exists a set of CFMMs on pairs of tokens, the routing problem is generally convex (without tx fees). Therefore we can suppose that MEV searchers can optimally route transactions through a network of CFMMs.

To study the impact of sandwich attacks on a network of CFMMs, we define Equilibrium routing. This states that a user’s trade routed over a network from a source token to a destination token over all paths is equal.

CFMM Pigou Example

How does optimal routing change when there is a sandwich attack on the network?

Assuming that there is no sandwich attacker of the $G_1(\Delta)$ path, then it would be intuitive to us that all things remaining equal we would split the trade through each path so that the price you receive for the token is equalized.

Assuming that $\alpha$ is the fraction of $\Delta$ that trades on $G_1$, then the equilibrium is satisfied by $\alpha^\star$ which satisfies

$$g_1(\alpha^\star \Delta) = g_2((1 - \alpha^\star)\Delta)$$

When both $G_1$ and $G_2$ have the same reserves $R_1 = R_2$ for token $A$ and $R_1' = R_2'$ for token $B$ then the equilibrium $\alpha^\star$ settles at $1/2$. The equilibrium can also be calculated by the maximizer of the following function

$$F(\alpha) = G_1(\alpha\Delta) + G_2((1 - \alpha)\Delta)$$

Where $G$ is the forward exchange function. In routing games, $F$ is called the potential function because the optimality condition

$$\frac{\partial F}{\partial \alpha}(\alpha^\star) = 0$$

gives us the equilibrium condition which is exactly what we found earlier

$$g_1(\alpha^\star \Delta) = g_2((1 - \alpha^\star)\Delta)$$

In the presence of a Sandwich attacker on $G_1$, the user also submits a slippage limit $\eta$, and as we know this quantifies the minimum output that a user wants from a CFMM. Therefore $\eta$ defines the sandwich attack $\Delta^{\text{sand}}$ as

$$G_1(\alpha\Delta + \Delta^{\text{sand}}) - G_1(\Delta^{\text{sand}}) = (1 - \eta)G_1(\alpha\Delta)$$

The $F(\alpha)$ with the presence of the

$$F(\alpha, \eta) = G_1(\alpha\Delta + \Delta^{\text{sand}}(\alpha, \eta)) - G_1(\Delta^{\text{sand}}(\alpha, \eta)) + G_2((1 - \alpha)\Delta)$$

and the equilibrium is

$$\alpha^\star(\eta) = \arg\max_{\alpha \in [0,1]} F(\alpha, \eta)$$

Essentially, what this means is when the slippage limit is set to 0, the equilibrium stays at $1/2$ because the sandwich attacker cannot capture any profit but as the slippage limit grows, the equilibrium starts shifting to $\alpha^\star < 1/2$. This means that the trader chooses to take a smaller fraction of their total trade on $G_1$.

The CFMM Braess Example

Before understanding the MEV example, a simple intuition for this can be built by looking at the Braess paradox that actually exists in game theory.

Road Network for Braess Paradox

Suppose that there are 5000 drivers that want to go from the start to the end. In the initial setup, no path from A→B exists, only A or only B are the possible paths. So a driver could take start → A → end or a driver could take start → B → end. When a driver takes A the time from start to A is the number of drivers on $a/100$ and then from A to end is 45 minutes i.e. $a/100 + 45$. When a driver takes B the time from start to B is 45 minutes and the time taken from B to end is the number of drivers on $b/100$ i.e. $45 + b/100$. As there are 5000 drivers, the fact that $a + b = 5000$ can be used to find out the equilibrium which is $a = b = 2500$. Therefore each route would take $2500/100 + 45$ which is 70 minutes.

Let’s assume that a path from A→B exists which has 0 time i.e. it is instant transportation, the first driver that does it would probably experience about $2500/100 + 2501/100$ time to go from start → A → B → end, which is about ~50 minutes thus saving the driver about 20 minutes. As more and more drivers take this path, assuming this number reaches 3000, the time taken for this path rises to $3000/100 + 5000/100$ which is about 80 minutes, and the people who stuck to the start → B → end path are experiencing $45 + 5000/100$ which is about 95 minutes. These people also start taking the start → A → B → end path and now since everyone is taking it, it takes about $5000/100 + 5000/100 = 100$ minutes which is much worse than the 70 minutes that people were taking earlier.

The Braess paradox is a classic game theory paradox which states that in congestion games adding resources may decrease the performance rather than improve them.

Inverse Braess Paradox in CFMMs

Now coming to our MEV example, as stated early on in the article the constant POA (Price of anarchy) can be interpreted as stating that sandwiches do not degrade routing quality. The paper constructs an inverse Braess paradox in which the presence of sandwich attackers in fact improves the network flow but the profit that the sandwich attacker can extract from the network is still bounded. The paradox proceeds in three steps.

1] Set $\Delta = 1$, in the absence of a trading pair between C→D, the equilibrium would settle to splitting the trade half between A → C → B and A → D → B, If we denote $\alpha^\star$ to be the fraction of $\Delta$, then the net output for $\alpha^\star = 1/2$ would be

$$\begin{aligned} G^{\text{net}}(\alpha^\star, \Delta) &= G_2\left(G_1\left(\frac{1}{2}\right)\right) + G_4\left(G_3\left(\frac{1}{2}\right)\right) \\ &= \frac{1}{2}1.5 + \frac{1}{2}1.5 = 1.5 \end{aligned}$$

2] Assume a pair from C → D is added, this additional resource as seen in the original Braess paradox problem, shifts the equilibrium such that the net output the user receives over all three paths reduces vs when no such path existed.

The equilibrium function becomes

and the solution for the equation for the new equilibrium is given by

and the equilibrium shifts from $(\alpha_1^\star, \alpha_2^\star) = (0.5, 0.5)$ to $(\alpha_1^\star, \alpha_2^\star, \alpha_3^\star) = (0.29, 0.41, 0.29)$ for the given diagram since the function for C→D is given by the marginal price function for Uniswap.

3] Now assume a sandwich attacker is added on the CFMM trading assets between C and D, the user specifies a slippage limit $\eta$, so over the path A→ C→ D→ the sandwich attacker computes the optimal sandwich (as we did in the case for Uniswap):

$$\Delta^{\text{sand}}(\Delta, \alpha, \eta) = \frac{-((1 - 2\alpha)\Delta + 2R) + \sqrt{((1 - 2\alpha)\Delta + 2R)^2 - 4(R^2 + R(1 - 2\alpha)\Delta)\frac{-\eta}{1 - \eta}}}{2}$$

The equilibrium now settles to

$$\begin{aligned} &g_2(g_1(\alpha^\star + (1 - 2\alpha^\star)\Delta) - g_1((1 - 2\alpha^\star)\Delta)) \\ &= g_4(g_5(g_1((\alpha^\star + (1 - 2\alpha^\star)\Delta)) - g_1(\alpha^\star\Delta) + \Delta^{\text{sand}}(\Delta, \alpha, \eta)) - g_5(\Delta^{\text{sand}}(\Delta, \alpha, \eta))) \end{aligned}$$

In a simulation with a sandwich attacker between C to D it is found that the net output increases until $\eta$ increases at which point, the equilibrium returns to $(\alpha_1^\star, \alpha_2^\star, \alpha_3^\star) = (0.5, 0.5, 0)$. Unlike the Pigou example, this demonstrates that sandwich attacker profit does not strictly increase with the slippage limit because rational users simply avoid the link the sandwich attacker is on, hence this is an inverse Braess paradox because instead of all users using the middle link, the existence of the sandwich attacker makes them avoid it.

Simulation for a sandwich attacker between C -> D

Price of Anarchy

The paper then goes on to formally prove this over the presence of sandwich attacks on a network of CFMMs. Constructing a graph $G = (V, E)$ where each vertex denotes a token (similar to the inverse Braess graph) and the edge represents a CFMM trading between $A$ and $B$. Each edge $E$ is also associated with a price function $g$ and corresponding forward exchange function $G$ that executes a trade.

Informally, it’s proven that

Conclusion

The paper is a formal description of understanding and expressing generic sandwich attacks against arbitrary CFMMs. The paper proves and computes bounds that depend on curvature and liquidity for sandwich attack profitability. It also establishes results in the reordering and routing MEV settings.

While the results have already been mentioned in the beginning, the paper also has practical impacts because it allows for a more formal framework for MEV searchers to benchmark and check their performance against the most optimal performance possible (can never achieve that because of the presence of fees). It also helps wallet designers to pick slippage limits i.e. suggest slippage limits to users while using DEXes so that their price execution is okay and not the worst possible.

Future work involves, extending this work to more complex forms of MEV including cross-chain MEV and MEV related to liquidations.

Understanding Aera Finance

Sat, 05 Nov 2022 19:01:22 +0000

A new era in DAO Treasury Management.

Since its initial summer in 2020, DeFi has run into various walls along the adoption path. Some of these have been user experience being too complicated; user interface not being intuitive enough, protocols not recovering from hacks, protocols needing scale which they probably can’t achieve actually to perform, ponzinomics which prop value but show unreal yield thus delaying “actual” adoption, the need for “trust” for protocols to become more capital efficient.

Capital Efficiency is one of the core problems in crypto. It sits at the heart of every mechanism design and is one of the most critical problems to solve. Why do we arrive at this problem? A thought occurred to me about the core problem of DeFi and general crypto a few days ago.

This tweet is no longer available.

x.com/i/status/1581894603883843584

One example of this is, let’s say you need a loan from a bank today, You walk into the bank, and the terms are put forth depending on the kind of loan you want to secure and the country you are in. In a country like India, you would end up furnishing around 12-14% for an education loan, while housing loans are about 8-10%. One thing you might be aware of, though, is to get a loan, you usually have to secure it with some collateral. This collateral is just a percentage of the loan that guarantees they can sell it if you default. The bank also has all your identity details, so they have proof of “person”.

All in all, this unlocks capital efficiency because these components build trust. In the on-chain world of crypto that is pseudonymous, any such faith is not possible (or has at least not been achieved in a scalable manner), which thus requires most loans and any such lending practices to secure their loan from 125% to as much as 500% in some instances. While there are protocols like goldfinch and maple which might be working on the problem, the solution involves off-chain KYC to offer these loans, which is an acceptable middle-ground for now but runs into problems at scale.

Almost every progressive step in crypto in the past two years has been about unlocking capital efficiency on some level.

Uniswap v2 → Uniswap v3
Liquidity bonds offer more robust liquidity for illiquid tokens
Rari’s permissionless pools collateralized at about 100% vs overcollateralized elsewhere
Goldfinch/maple’s attempt at undercollateralized loans
Perps on chain moving from the Synthetix model to DyDx to GMX etc.

Where does all of this connect with treasury management, though? Treasury management is an underrecognized problem in the DeFi landscape, and few have tried to solve it. It is well-known that almost all protocol treasuries primarily comprise the protocol’s native token. This makeup of a treasury also leads to value erosion of the treasury in case of downturns, and sometimes this can also mean the project’s death. One of the most significant raises in 2021 was by Fei Protocol which raised about 1.3B $ in its genesis event. While it might have been marred with some issues early on, the DAO treasury was never managed to its fullest and was left to the mercy of the market and was in a constant downturn ever since the Crypto top in Nov 2021.

Fei also merged with Rari Capital, a lending protocol to synergize the stablecoin offering with lending to increase their capital base and make it more efficient. Rari got hacked in April and exploited by a reentrancy attack in April for 80M. Fei’s treasury was ultimately insufficient to repay the entirety and make the people who lost the money whole. The Rari hack was also a reason for Babylon finance, another protocol, to shut down. Fei Protocol ultimately winded down its operations around September 2022.

Joey @ NS @joeysantoro

Replying to @joeysantoro

Rari is an ideal platform to enhance FEI demand and utility, as well as a place for PCV to grow via the upcoming Vaults 📈

FEI liquidity as a service provides the oracle and liquidity to support any trades Fuse users may take 🌊

Nov 16, 2021

DeFi protocols, in rare events, do not have a backstop to protect themselves or their users or ensure the sustenance of a project. Defi protocols are not FDIC insured, which in the case of TradFi banks, gives some relief and acts as a trust mechanism for depositors in case of some rare event that takes the bank under and the users are made whole to an extent. A common practice in the case of Centralized Exchanges in crypto is to keep insurance funds from their fees and otherwise, which is utilized in the case of hacks or other extreme events, like the Binance SAFU fund, something similar is also maintained by BitMEX; Deribit etc. Deribit, in a recent hack, covered losses using its company reserves. This does not include rare events of liquidity cascades and rapid drawdowns of assets, which sometimes leave these protocols with bad debt.

DeFi Treasury management is challenging since many treasuries are filled with native tokens from the projects. A lot of pushback occurs whenever there are mentions of treasury swaps to secure stablecoin for operations or other required operations—for example, the Love-001 Maker DAO Proposal, the Sushi DAO proposal for a strategic raise etc.

Although in the past year, a variety of protocols have slowly started garnering attention and gaining traction, they have not achieved sustained high volume usage and barely pass a few basis points of the total asset value that they cover (e.g., you would imagine that a successful options protocol would at least have a few billion dollars worth of eth If it were successful). This usage of products differs from traditional finance because corporations tend to hedge products they offer using derivatives, options or other exotic products. Hence, this leads to the products having some continued usage (which is non-speculative).

Total Treasury Value - DeepDao.io

Top 10 Treasuries on 3rd November - DeepDao.io

DAOs and Allocation Problems

So the institution of scale that exists on-chain for utilizing these protocols are Decentralized Autonomous Organizations (DAOs). DAOs are not profit-maximizing entities but might instead have a variety of objectives they might want to look at or achieve. In that case, they are similar to entities with some goals and hand over the money to “professional” services that manage these funds, like venture funds or hedge funds.

DAO funds need to be managed for a variety of reasons:

The DAO treasury might want to maintain an insurance fund to cover the shortfall from their liabilities. In this case, the DAO has to ensure that the insurance fund’s assets are worth more than the liabilities.
The DAO treasury might want to secure the “runway” of the protocol, thus ensuring continued research and development.

Unlike Traditional ways of adherence to terms secured by the law, on-chain DAOs have to resort to smart-contract calls through governance proposals to do anything that involves their treasury. For example, if Uniswap wished to convert some of its treasury in UNI to USDT/DAI/USDC today, it would first launch a proposal and then access the funds. This ultimately makes you think that liquid funds are essential for an ideal and suitable allocation protocol (in case the protocol needs to remove the funds in case of rare events immediately), and the interaction required to deposit or withdraw should be kept at one smart-contract call each.

So let’s think about this for a second. We have DAOs who have treasuries that could be optimized better, and we have DAO stakeholders, i.e. token holders who hold the protocol token and might be opposed to any sale of assets since that erodes their value. We may have analysts and speculators who could be outsiders or funds invested in the protocol who think the DAO treasury can be managed better, thus managing protocol risk and ensuring the project’s existence. We have multiple parties whose incentives need to be aligned.

Decentralized Allocation as an Adversarial Optimization Problem

In this framework, we formulate the problem in this way, we have

$W$ → The wealth of the asset allocator
$\pi$ → The portfolio which allocates $W$ to a variety of assets
$N_{\text{assets}}$ → All the assets that are possible
$\Sigma$ → Onchain state of the protocol
$\Sigma_t$ → Onchain state of the protocol at time $t$
$f$ → A convex objective function that determines asset quality, essentially the function you want to measure the performance with; you want this function to be maximized
- $f(\pi, \Sigma_t)$ → The parameters of this function $f$ are the portfolio $\pi$ and the onchain environment $\Sigma$, which can also be $\Sigma_t$, i.e. the environment at time $t$, the function would take in these parameters and output a measure of asset quality.

So in this formulation, we have an asset manager whose job is to use the wealth $W$ and allocate the portfolio across assets. In most cases, this simple asset allocation would be like active fund managing, where depositors deposit money, and the asset allocator gets a fee based on performance. Still, in our case, the wealth $W$ belongs to a DAO managing a protocol. The onchain protocol’s state might defer at different points in time, so the context of managing these funds must be adjusted to the protocol’s needs. For example, as mentioned earlier, if a lending protocol wants to hedge its outstanding loan book, its on-chain state is of prime importance since it is dynamic across time. Additionally, you have to allocate assets in such a way that the potential shortfalls are addressed.

So thinking of it as an optimization problem, you are searching for the optimal portfolio, which is the best possible portfolio given your protocol and various possible combinations of the portfolio across the variety of assets available

$$ \pi^*(\Sigma_t) = \sup_{\pi} f(\pi, \Sigma_t) $$

where $\pi^*$ gives the optimal portfolio.

This formulation is perfectly valid in case there was just one fund manager/analyst/speculator who had the power to call all the shots. In the case of a decentralized organization setup by design, multiple people have the ability to suggest and request changes.

So now you end up thinking about the problem in this way, person A might be a token holder who owns the protocol token $X and believes that all the stablecoins in the treasury should be used to purchase the token $X, person B might have a short on $X and in some way pose as someone who wants to ensure the “runway” of the protocol and suggest selling all the $X assets or at least a few million dollars which leaves him profitable but seems benign for onlookers. When you think of the possibilities like this, you end up with $n$ possible people who suggest changes, and each has their own objective function that they might be looking to maximize or profit from.

When you try to optimize multi-player optimization functions like this, it is likely that the optimal portfolio that the DAO might need does not intersect, i.e. is not the same as the ideal scenario for any of the participants who suggested changes to the asset allocation.

Another thing to note when on-chain DAOs are considered is the limited ways of portfolio rebalance because, to maintain transparency, there’s a predisposition to use on-chain mechanisms like AMMs, gnosis auctions or lending protocols (which can also involve using vaults from other protocols in some cases). But coming back to our earlier example, let’s say there’s a person C who wants to buy the token $X cheap, so suggests a treasury adjusted of selling all tokens to $, in case person C is an arbitrageur (someone who trades between onchain and off-chain marketplaces to profit from price divergence) then he would prefer large asset changes since that would present him a more significant opportunity to benefit in multiple ways. Therefore participants who suggest changes should have some skin in the game and accept some risk against any portfolio changes they offer.

Finally, the on-chain state of a protocol is subject to constant change. This would also mean that DAOs are best served when there are regular, and good portfolio rebalances, which optimize their position and ensure that the protocol is well-placed to handle any adverse events. This would require participants to suggest these changes at regular intervals. Incentivizing portfolio submissions is the best way to make this possible and, coupled with some skin in the game from the portfolio submitters, ensure high-quality submissions.

The design of a decentralized allocation protocol would thus need to account for the following:

All actors are adversarial: A good design principle which applies everywhere is “design for extremes, and the middle will take care of itself”. Assuming that every actor is adversarial is an extreme that the allocation protocol should consider, so it progresses despite diverging/malicious wants of agents.
Actor Collusion: As possible as adversarial actors are, multiple submitters can submit different portfolio directions and conspire to make inefficient allocations but yet profit from the same (arbitrageur collusion for profit by which each suggests huge asset allocations across different assets). Ensuring that collusion is a zero-sum game is important.
Submitter Incentivization: Pay submitters with respect to asset performance measured by reputation or contribution to ensure the quality of submissions and avoid submissions of portfolios which suggest random asset allocations to maintain the appearance of submissions.

Aera Finance - A decentralized allocation protocol

Aera protocol constructs a competition amongst agents who submit portfolios for treasury allocations. These agents are known as vault guardians. The competition is structured as an iterative skill-based model to maximize individual rewards. The DAO pays for these rewards.

Vault guardians have to stake assets that can be slashed if their Allocation causes the DAO to suffer a loss. The “slashing” mechanism is outsourced to arbitrageurs by allowing them to participate and suggest improved treasury allocation. They spot price discrepancies between the DAO portfolio and the global market. These arbitrage losses the DAO treasury faces are passed via slashing onto the vault guardians if the losses are sufficiently large.

Aera finance has a crucial feature which is time-scale separation between different participants. In other words, a DAO depositing money will have a very long-term view of their money and wouldn’t be prone to constant deposit and withdrawal due to the nature of their existence. At the same time, vault guardians, since they have to keep the on-chain environment in mind and risk slashing through arbitrageurs spotting price discrepancies, are in some sense forced to suggest changes in periodic intervals, thus constantly trying to achieve an optimal portfolio. It is the DAO that authorizes and selects the optimal Allocation among submissions.

The Allocation is done as follows:

A DAO treasury can be interest a small subset of all available assets (subject to changes, preferably infrequent)
These assets can be stablecoins, options, loans, AMM LP (Liquidity pool) shares, or any other possible ERC-20 representation.
Different guardians propose portfolios per DAO preferences and mention the percentage that the DAO treasury should be changed to different subsets.
The protocol then constructs a single portfolio taking all the submissions into account, and the DAO executes it.

The aggregation rule that Aera uses is a weighted median aggregation rule. It’s similar to the one used by the Pyth network (an oracle protocol on Solana). While better aggregation rules have been researched and found they are not feasible given the computational limits of blockchain. The aggregation rules also naturally correspond to the reputation scores of submitters; thus, higher reputation submitters have more weight.

How are the assets actually deployed? How are the guardians slashed? Once the single portfolio from all submissions is constructed, Aera puts the assets in a CFMM while ensuring relative proportions as arrived upon. The DAO will act as an LP to the CFMM that executes the trades for their treasury portfolio adjustment. Arbitrageurs can trade against the portfolio until the targetted asset proportions are reached. CFMMs incur an impermanent loss on LPs. Therefore, if an allocator submits a portfolio that deviates too much from the current state, this increases LP loss due to arbitrage. And this loss can be passed onto the guardian’s staked assets through slashing.

How are the portfolio submitters graded and incentivized? Portfolios submitted by participants are graded on their contribution to how much they change the final portfolio. Suppose adversarial/malicious portfolios are submitted which deviate from optimal values and take the portfolio in the other direction. In that case, it makes natural sense to either give them much less rewards or even slashing their staked assets in extreme cases.

Grading for participants

Another part of the grading also involves getting grades of each submitter to construct relative ranking in order to determine how much of the fees is distributed to whom. Shapley value is a concept from cooperative game theory. It essentially describes the contribution that each player makes to the overall cooperation/plan. In our case, the Shapley value of the ith player would represent how much player $i$ contributed to the final portfolio. We get these Shapley values by constructing them from individual grades. The following equation gives the total fees spent by the DAO on block height $h$ where $i$ is the ith player, and $\lambda_i$ gives the Shapley value of player $i$.

$$ \gamma(h)_i = \gamma(h)\, s_i(h) \left( \frac{\max(\Lambda_i, 0)}{\sum_i \max(\Lambda_i, 0)} + \frac{\min(\Lambda_i, 0)}{\sum_i \lvert \min(\Lambda_i, 0) \rvert} \right) $$

Essentially, every player is given pro-rata fees/rewards relative to the set of all players who have contributed positively to the portfolio.

Comparison to Futarchy

There are some similarities between the solution offered by Aera finance for treasury management and Futarchy. Futarchy is the mechanism of executing governance decisions via prediction markets. The significant similarity in both is that participants (in our case, participants who submit portfolios) to succeed mostly need to make non-trivial calculations which sometimes involve asymmetric information. Another similarity is the usage of Automated market makers in both instances.

While there are similarities in how they formulated Futarchy and Aera, Aera vaults address very different markets and have other fundamental differences in how they operate. Futarchy needs an entirely new market to be created from scratch, while Aera offers the treasury management solution to existing DAOs with liquidity. This existence of a market allows Aera to bypass futarchy criticisms like the cold start problem, lack of liquidity etc. Also, it’s not just the existence of organizations but also the presence of participants, i.e. submitters and arbitrageurs, which makes offering such a solution easier.

Unlike LMSR (Logarithmic market scoring rule) used in the case of Futarchy has not seen much adoption beyond niche elections, Aera finance uses CFMMs, which have gained a lot of acceptance and traction and manage billions of dollars in liquidity today.

Time-scale separation is one of the essential features and also a difference between Futarchy and Aera. In the case of Futarchy, the participant can participate and hedge exposure until up to the last moment, and the payoff is binary. In the case of Aera, as we saw earlier, portfolio submitters are forced to keep the on-chain environment and the protocol status in mind and suggest periodic improvements/changes to the portfolio allocation for optimal performance and to keep the treasury allocation market aware.

Finally, Aera is fully on-chain and cares only about the on-chain environment and state, unlike prediction markets which generally try to solve the “oracle problem,” i.e. bringing off-chain information to an on-chain environment. Aera overcomes the significant problems of Futarchy, as mentioned above but also deals with the whale problem.

Insurance Funds in Crypto and Treasury Management

Insurance funds are a very contentious topic in crypto. There have been a variety of attempts at insurance protocols in crypto like Insurace, Nsure, Bridge Mutual, Risk Harbor, and Nexus Mutual. Some of these allow users to deposit a portion of their earned interest which pays out in the case of a liquidation event. Insurance was and is a much-desired protocol on-chain. Protocols have attempted to gain traction because it would be like an on-chain savings account, much like FDIC insurance, as earlier mentioned in this article.

Most Insurance protocols, though directly, try to offer these insurance services to the users. In the last year, there have been a myriad of hacks that would have benefited from insurance on their assets. Some audit agencies provide insurance on audited code to a certain extent (10M $). Think about this for a second: User experience for insurance on the chain involves you manually insuring every position of yours in one of these protocols, which is like opening an account and then visiting an insurance agent for every financial account you might make. In TradFi, the institution directly purchases or holds the insurance on customer fund losses.

One way of achieving an Insurance fund in Defi is to use a protocol’s stake-based insurance fund and rebalance it to achieve the required criteria. For example, Aave and synthetic utilize stake-based insurance funds. The stakers are rewarded with the protocol’s native token. Still, if the system has liabilities, the insurance fund’s assets can be sold to cover these liabilities.

The important part here, though, is the rebalancing to uncorrelated yield-bearing assets so that the insurance fund can handle rare events better. This also avoids overconcentration in a single asset. For example, let’s say a protocol had a token $Y and an insurance fund comprised of $Y, ETH etc., and the protocol operated a lending market on ETH. Let’s say there was a massive selloff in ETH, and the protocol’s lending market for $Y was exploited to some extent by oracle manipulation. If the insurance fund is denominated in the same assets, then it will also face the same drawdown that the assets it lends face. Therefore, it is of prime importance to diversify or at least remove the correlation between assets and liabilities. This rebalancing of the insurance fund to match its liabilities or hedge downside risk is called Risk-aware treasury management.

Aera Finance uses vaults, i.e. smart contracts that can assist DAOs with treasury management. Vaults have gained immense popularity in crypto for various uses. It has become a commonplace offering for all kinds of protocols. These Vaults employ various strategies involving rebalancing or distributing assets into different protocols, which might offer yields on the vaults’ tokens. The vault, in essence, becomes an automatic rebalancing mechanism where the user gives the money and doesn’t have to worry about the strategy itself. One issue with vaults is that the transparent and public usage makes them prone to adverse selection, and most vaults right now optimize for yield maximization and haven’t explored more complex objectives. In the case of Risk-aware Treasury Management by Aera, a DAOs utility function is expressed as an objective function through code and is then used to grade submitters and determine their share of fees.

What is the duration between two subsequent rebalances of the vault? The question is hard to answer. Ideally, you want the vault to be volatility aware so that in sudden downturns, the vault tries to rebalance in favour of better assets or convert to stablecoins to incur fewer losses. Aera optimizes rebalancing by using a two-speed incentive mechanism with both a fast and slow path.

Slow path → Involves parameter submitters who submit a portfolio to the vault infrequently.
Fast path → Involves arbitrageurs who trade against the vault to rebalance portfolios.

As seen earlier, collusion between both parties is avoided because any huge loss is incurred at the other party’s expense.

The final piece of this puzzle is incentivizing various participants. Submitters are crucial to the ecosystem, so rewarding them makes sense. But Parameter/portfolio submitters need some skin in the game and are hence required to stake “a sufficient quantity of capital” to make submissions, this could be an order of magnitude or two lower than the size of the vault if they are malicious actors then their submissions cause adverse outcomes or losses for the vault and this is, in turn, passed onto them by slashing their staked assets.

Submitters are incentivized to enter the portfolio contest only if they have sufficient confidence in their ability to predict suitable parameters. At the same time, rewards will be better if their influence on the performance of the treasury is higher as well. One crucial thing monitored is submission quality over time for submitters. The highest quality submitters over time should be rewarded and recognized. Aera calculates quality scores for submitters to keep track of this. The incentive of DAOs to use this service is to have better treasury management and avoid losses also because DAOs usually don’t have active asset managers and would like to avoid that by just focusing on protocol development and other stuff. Depositing DAO treasuries in Aera opens up submissions from best-in-class asset managers.

Aera has a running single submitter N-token vault, the specs for which are as follows.

In a future article, I will cover how quality scores are measured, how exactly protocol risk is hedged, and the considerations you make in objective function selection.

Aera finance is a solution for a critical problem in DeFi and, if successful can become the status quo for treasury management practices for DAOs and protocols while keeping risk in mind and not having to worry about market conditions impacting their treasury value.

The Artgobblers Renaissance

Sun, 30 Oct 2022 20:35:28 +0000

NFT Distribution paradigms, Complete on-chain games and a potentially new meta.

Art gobblers is probably one of the most exciting projects coming out this quarter of the year. Several different components have slowly led up to it, so just a “short” article looking at various aspects of this project and probably another NFT blue-chip is you are to believe the echo chambers that I am a part of.

A brief view of Crypto Distribution

One of the core problems in crypto, whether it be the tokenomics of a project or even in the NFT space, is the question,” what is a fair distribution of a supply?” Put in other ways, how do you distribute things in the right way (and what is “right”)? While when it comes to airdropping a supply of tokens, there is no correct answer, and a search for a good answer continues, you would think that this might be somewhat different in the case of NFTs. Regardless of low supply dynamics, if there’s something common in the general crypto landscape is an eventual accretion of supply in the hands of a few whales or super-collectors. Most people seem to agree that supply should be in the hands of more people than fewer, at least when the distribution starts.

In the case of token supplies, a lot of this can be traced back to Sybil practices in the case of airdropped supplies or simple whale acquisition during distribution phases, i.e. when the coin dumps a lot and the whale just buys a considerable part of the supply. But more common than not, even the genesis distribution of most token supplies looks much more skewed than you would want them to be, irrespective of various rules/anti-Sybil practices that might be in place. Of course, some of this can be attributed to the fact that sometimes core “contributors” or early believers (other than the team itself) of a project can be chosen to be airdropped a more significant piece of the pie because they’ve been with the team since the start and this is just one way of thanking everything they’ve done.

Now that my short digression ends, how does any of this compare to NFTs, and what does this have to do with art gobblers? NFTs have had a variety of ways of genesis distribution, they originally started with crypto punks which was a free mint back in 2017/18 (I think), and in this past cycle, it started with hashmasks choosing the bonding curve, which went on for quite some time, then started free mints again which were quite possibly some of the worst ways of distributing assets because it devolved to on-chain gas wars noticeable if you follow the ETH gas fees in any year the spikes mostly corresponding to NFT mints rather than crashes, these free mints somewhat also get exploited by bots but in the free wild forest that we describe our chain environment as anything seems to be fair game once the rules of a “competition” are announced.

Artgobblers starts with a whitelist free mint which on some level ensures a much broader distribution base even if we see the supply accrete in the hands of a few and this other mechanism which will allow Artgobblers to be minted for the upcoming decade, VRGDAs (Variable rate Gradual Dutch Auctions).

A briefer rant on the state of NFT supplies

One of the most misconstrued and ill-thought parts of NFT games and design has been how NFT distribution has occurred and how it doesn’t allow for the mass adoption of a game.

I remember this time when I wanted to apply Axie Infinity last year around June; I bought an Axie for 0.2 eth and started the game thinking it would be enough and was displeased at the sight that I required three Axies to play the game (Ya I’m probably a poor pleb) which would require me to buy it for almost one eth in total (my luck was such that I found the game just before its run to being a billion dollar game was starting), A few thousand dollars to just have the taste of the game is something that no gamer will ever pay even if it were the most anticipated game in history.

In an attempt to embrace the ethos of non-fungibility, on-chain game creators misconstrue or leave simple ideas from general game design behind, i.e. to make it accessible for as many people as possible by gating access behind the need for an NFT, you sort of announce that you are just making this game for the few hundreds or at max thousand of holders that your NFT collection has. It is pretty surprising to me that almost no one ever talks about this caveat when talking about web3 games.

Most crypto “game” developers sort of offset this argument by saying that NFT collections are like luxury skins/luxury collectables to give them a better taste of the game while they might release auxiliary supplies which would still allow for access to the game but in a more “standard” edition instead of the limited edition manner that the OG collection of the said game might have. The issue with any such idea is that you immediately drive a wedge in the minds of those playing the game that you are making a play to win games where skill might not matter, or you might not have a shot at getting better “loots” or digging rare stuff with your standard edition NFTs. A lot of game developers in the web3 space seem to pride themselves on saying that their smaller 10k collection will be OG and will have unique features when the game will be launched, including boosted drops, better stats etc., which might help them boost their sales but will probably never help their adoption. In retrospection, a few years later, I think this is something most people will notice and try to address (just my guess).

Now that I’ve motivated the problem, what are the different ways of solving this problem? As mentioned earlier, you make additional less valuable collectables with infinite supplies, which allow access to the game by anyone who owns it, thus also reducing the upfront cost of access, this has been tried by the loot, mloot projects last year. Another way might be to allow an infinite or a large finite distribution but start with a limited supply and gradually inflate supply, thus allowing everyone to be on equal grounds and not making them feel they own something less superior but not overextending your supply (this idea is very good for beta-testing to the main launch of a game because you can slowly stress test your system while adding players systematically). How do you price or release these NFTs? This is where VRGDAs come in.

Note: Not that this might not be entirely true in the case of Artgobblers itself, but VRGDAs are a promising way of steadily growing supply and pricing it through supply and demand. At the same time, there might be instances where you might want to choose a fixed cost of access and keep releasing X amount every day at the same cost. VRGDAs are a promising alternative and probably an ingenious way of NFT supply distribution. Artgobblers itself is just starting with a 2k mint and will have 8k more over the next decade. Anyone who knows on-chain greed probably knows that it will be priced at a level out of the reach of most people who might want it; even then, it is an interesting experiment to study what happens when you grow supply beyond initial mints in low-supply assets. Digidaigakus, a recent project with a 2k mint, are currently priced at 10eth and went up to as high as 20eth but was not as hyped as Artgobblers, so if it achieves the blue-chip status, a 2k supply achieving a 100eth floor, is not something wild. (While price prediction is not my milieu, Just doing this cost analysis to show how out of reach the collection might be to people).

Auctions, Dutch Auctions, Gradual Dutch Auctions and Variable Rate Gradual Dutch Auctions

Auctions have found a very prominent application in crypto, mostly in places like Foundation, Superare, Makersplace, and Zora, for creators to sell 1/1 pieces they make. Auctions are a very good form of gauging the demand for a particular asset and help in the total price discovery of an asset. In the case of 1/1s, almost every popular art piece in history has been sold through auctions at sotheby’s, christies by various collectors.

Usually, asset auctions occur in a way where a bidder first bids and then other bidders outbid until the top most bid wins in the case of physically settled auctions, we see the auctioneer calling out the top bid and asking if anyone else is present in the case of online auctions like that we see in 1/1 NFTs a reserve price is first set and then about a 24-hour onchain auction begins and whoever has the highest bid at the end of it wins the auctions).

A standard English auction on foundation, Artist: yueko

Dutch auctions, also called descending price auctions (more self-explanatory this way), are a type of auction in which the auctioneer starts with a very high price and then gradually lowers it stepwise until someone places a bid. The first bid which is placed wins the auction, thus avoiding any further bid wars.

Dutch auctions, in the case of financial assets, have a slightly different form where in everyone submits a bid, and then a price to sell it at the highest value is arrived at. This value is arrived at by taking the cut-off price, i.e. the lowest successful price, and the remaining money is returned to everyone above the threshold. Simply looking at an example, If there are 200 bidders for 100 shares and the 100th bidder bid about 50$ and the 1st bidder bid about 1000$, the cutoff price, in this case, is 50, and that’s the value the entire auction settles at which the top 100 bidders pay. Although Dutch auctions have found their place in crypto, they are too efficient compared to those seen in traditional markets because onchain auctions are entirely public, so price discovery is complete, while traditional markets have a more predictive nature, leaving some scope for further price discovery. There are no sealed bid auctions onchain currently available on-chain.

Dutch auctions are not great for big supplies because price discovery occurs pre-mint while the expectation of a rational actor might prefer price discovery to occur after. This take definitely discounts those collections which might be low-key or have some announcements post mint there in some information, in this case, is not priced in.

Gradual Dutch Auctions, GDAs, was a mechanism designed to sell “illiquid assets”, in our case, low supply NFT collections. While both discrete and continuous GDAs over various functions exist, one way of thinking of Gradual dutch auctions is that the sale of tokens happens in batches where each batch is a regular dutch auction settled at some price. Every incremental auction starts at a higher price. For simplicity, a batch size of 1 gives you an intuition that every time period $T$, a new NFT auction is conducted, so over time, the price discovery is smoother instead of settling all NFTs at the same price. How is this price determined? While a variety of functions could be applicable here, this is one such choice.

From the GDA blog: https://www.paradigm.xyz/2022/04/gda

$n$ here is the number of NFTs sold, $\alpha$ is the scaling factor, $\lambda$ is the decay constant, $t$ is the period, and $k$ is the initial price set. If you set the decay constant to 0 and, the initial price to $k=1$, the scaling factor to 1.1, then the auctions should be priced 1, 1.1, 1.21 etc. Another example which shows the cumulative sale cost from the blog

One drawback of GDA auctions would be the inflexibility of scheduling the price of these auctions, where every auction starts at a higher price while being unaware of supply/demand dynamics. Let’s just say you had some schedule for the supply increase in mind; simple GDAs would not be very helpful in this case, and while they would work you could see a longer turnaround time for auction completion.

Variable Rate Gradual Dutch Auctions (VRGDAs) are schedule-aware Gradual dutch auctions where in projects could release supply according to some distribution curve they have in mind. Simply put, from the blog:

Variable Rate GDAs (VRGDAs), designed for Art Gobblers and used in 0xMonaco, let you sell tokens close to a custom schedule over time by raising prices when sales are ahead of schedule and lowering prices when sales are behind schedule.

Schedule aware pricing

Such schedule-aware pricing increases the price of the tokens if more supply is released ahead of time and decreases the price if a lot of the supply is unsold, thus making it easier to buy tokens and thus trying to spur interest. The ingeniousness of this design mechanism is simple demand/supply price dynamics directly priced in through a price function. (I’m a fan). A variety of release schedules are thus possible, e.g., linear, square root, or perhaps triangular can also be considered an interesting experiment.

Art gobblers

Gobble gobble.

The Artgobblers renaissance isn’t about what I have covered in the article, but it’s about bringing it all together in the form of a “complete” on-chain game. Complete as in the game will be released in its final form, and no further developments will happen from the team. It’ll be a complete NFT project (rare given how most NFT projects count on “roadmaps” to secure further funding or spur more interest in their project, the only exception to this probably is Cryptopunks, Ethereum’s first NFT project, which is more a cultural phenomenon in crypto now).

So what is the flywheel that Artgobblers is trying to set in motion to find its place as a cultural collection central to not just crypto but perhaps beyond crypto as well?

The Draw Tool

The draw tool was released in august for artists to draw various art with support for iPad and desktop. The draw tool also records the entire process of creation so you can take a look at the entire process when you are done making an art piece, akin to time-lapse recording in autodesk sketchbook on iPad and other similar apps.

Art Gobblers @artgobblers

Attention! Our experimental Draw Tool has just been beamed down to the Earth’s surface. We seek willing guinea pigs to test its functionality: artgobblers.com/draw/

Aug 16, 2022

Bringing it all together

The draw tool allows for the creation of art and while might see some improvements in the future (not counting on it), is a complete app in itself which has seen some pretty cool art pieces come of out of it.

Art Gobblers @artgobblers

PennyGobbler by @CALICOJACK69
artgobblers.com/drawing/Jxr8rak

Oct 30, 2022

Once ArtGobblers goes live on 31st October, art made through the draw tool can be minted as a 1/1 NFT on “pages”, another NFT which is a part of the Artgobblers game. This is called “glamination”. Pages are also scarce and are released through VRGDAs just like Gobblers themselves.

A short note: ArtGobblers will be a 2k initial whitelist mint, and the rest of the 8k will be released through VRGDAs over the next ten years while pages are infinite, only 69 pages are released daily slowing down to about 10 per day over time. For more details, refer to the original blog.

Glamination can only be done on empty pages, and these empty pages can only be minted with GOO. GOO is an erc20 token which will be emitted by the Artgobblers.

GOO is a token for the in-game economy of the Artgobblers ecosystem. It has two uses, it can be used to create more gobblers or mint more pages. Gobblers can gobble the pages, at which point the art becomes a part of them.

From a collector’s perspective, a gobbler is a museum NFT that you can own to store your favourite art pieces that have been minted through the draw tool. Regardless of the popularity of the NFT collection itself, the NFT space, in general, has seen a considerable uptick of collectors of 1/1s who have showcased their NFTs in various ways owning spaces in virtual VR spaces or otherwise. An NFT of NFTs such as ArtGobblers is probably the first of its kind but not the last. NFT museums might be here to stay.

The game of GOO, an erc20 token that will be emitted by gobblers, will probably have the attention of the collectors for the next few weeks. As has been common in the space, erc20 tokens attached to NFTs have acted as pseudo-price discovery mechanisms and an alternative way of finding a market cap for such collections, an important distinction made by the team before hand, in this case is GOO isn’t meant to be anything beyond an in-game token with specific use only for collectors and other players in the Artgobblers ecosystem and is pretty much valueless to speculators (which might not stop people from going after this relatively low supply token that it might be for the first few weeks to months).

t11s @transmissions11

an inflationary utility token with specific in-game uses, very important not to get carried away

paradigm.xyz/2022/09/artgob…

Sep 13, 2022

GOO is emitted by Gobblers (as I’m probably writing for the third time), and every owner will have a constant production of this token, so owning this without a gobbler essentially means you are at risk of dilution. (A very similar mechanism was found in OHM and OHM forks last year. Ideally, you owned a certain percentage of the supply, and then you staked it, and there was constant issuance of ohm by which you would always own that percentage of the supply, the keyword here being ideally since other dynamics at play almost always ensured that you did get diluted, so it quite didn’t work out that way).

In the case of GOO, every gobbler has a goo issuance, and every gobbler has a goo issuance multiplier. Owning multiple gobblers with different multipliers is the same as having one gobbler with the sum of those multipliers.

This multiplier dynamics of goo issuance at play gives rise to different strategies. One such recollection by a beta-tester, Misaka:

This tweet is no longer available.

x.com/i/status/1585032905516257280

The dynamics involve variables which involve thinking about how much goo you make, how you pool your gobblers to maximize issuance and the fact that Artgobblers can only be minted with GOO and that empty pages can also only be minted with goo. Some highlights from the thread around these dynamics are

everyone was in a prisoner dilemma and kept pushing the price of @artgobblers up.
You could acquire a legendary gobbler by burning a huge amount of @artgobblers in exchange for a Gobbler with 2x the total Gobblers’ multiplier you burned.
The price of @artgobblers quickly went from 10k to 24k $GOO in 2 days after launch

Ultimately it was found that people who had gobblers would probably end up colluding/ making by coalitions to ensure/maximize their production and have a better edge for pricing/acquiring gobbler NFTs. This was something also found in Spearbit’s audit of Artgobblers

https://github.com/spearbit/portfolio/blob/master/pdfs/ArtGobblers-Spearbit-Security-Review.pdf page 18

The entire gameplay in a chart looks something like this

https://twitter.com/AdamHustles/status/1584326159231176704?s=20&t=9gEgRl-vu34JH1xQg9X2WA

In a recent question to t11s, he mentioned that Artgobblers could gobble any art that is fed to them i.e. any ERC721/ERC1155, which would probably mean that all Artgobblers are NFT museums from the day they are minted. This would only require further support from someone in the community to display these since their front end will only support the pages. This way, once the initial few weeks of Artgobblers are over, they will pretty much become unique 1/1 museums depending on the collector that owns them, which is quite an exciting experiment in the NFT collectable landscape.

t11s @transmissions11

Replying to @0xemperor

At the contract level, Gobblers will actually gobble any ERC721 or ERC1155 NFT you give em.

We plan to only show draw tool generated pages on our frontend, but would be sick to see community frontends that highlight other types of gobbled NFTs as well!

github.com/artgobblers/ar…

Oct 31, 2022

ArtGobblers, if it succeeds, will probably find itself only second to the marvel of Cryptopunks in becoming the zeitgeist of NFTs, especially those involving onchain games. It would also serve as a playbook for people who wonder if every onchain game has to be an elaborate MOBA/MMORPG or something else. It would vindicate the idea that token supply and collection dynamics can also serve as a simple onchain game.

The ArtGobblers renaissance starts once the mint is complete on the 31st of October, and we then have a lovely study of various onchain dynamics and possibly the start of a small niche cultural movement in crypto. My bet on it lasting long due to degen collectors in crypto and defining collector zeitgeist for a few years or perhaps decades to come.

Understanding Sudoswap

Wed, 17 Aug 2022 13:54:53 +0000

Non-fungible tokens have been used in various use cases since their inception. The ERC-721 standard initially finding its use in art and then mass Pfp collections. A trend started in Jan 2021 with hashmasks and found quick popularity with various subsequent collections. Many agree that NFT narratives dominated the cycle in the latter half of 2021. We saw art blocks, punks, apes and other derivatives so on and so forth.

Where does one buy NFTs? The available marketplaces include Super rare, Zora, Foundation, Rarible, Opensea, Looksrare, and NFTX. The first 3 are marketplaces for 1/1 art pieces, Rarible never found its groove or capitalized on the relatively early mover advantage they had. Lookrare didn’t exist until earlier this year, and while NFTX was a novel product, it didn’t precisely capture the market demand that is usual to marketplaces (which it wasn’t exactly meant to be). Nftx essentially unlocked the index idea for NFTs, wherein you didn’t have to own the entire Nft and could own fractions of it, thereby maintaining exposure to the tokens you like/blue chip Nfts.

Opensea has found its place as the king of marketplaces with its superior discovery and relatively easy-to-use platform and clocks in about a 100 million volume every week, reaching as much as 1.5 billion dollars at its peak, and has had about 2 million unique users and much more. At the end of the day, though Opensea is still a traditional marketplace experience and isn’t anything crypto native, its simplicity and ease of use have found a loyal customer base.

There haven’t been many crypto-native (not just in origins but ideas) solutions to the NFT marketplace problem. The attempts have gone awry in various ways, such as misplaced or hyper incentivization, which has led to false growth of revenue or volume but hasn’t led to sticky usage, or relatively niche-r products which have hit the ceiling on their growth arc.

Sudoswap is a new, not just in concept but in execution as well, NFT marketplace. A promising difference from the likes of other marketplaces which have just tried to import the Opensea playbook onchain. Sudoswap uses Automated market makers (custom-tailored to NFTs) to allow NFT selling and buying.

Background of various models for tokens

Order book models are usually the most efficient when it comes to trading tokens, stocks etc. The CLOB (Central limit order book) model, as usually seen in crypto in centralized exchanges, offers a way of efficiently selling your tokens at a particular price (the fancy word being tick), i.e., if you want to sell a token at 1$, you can place your order at the same and often have increased levels of tick granularity from 0.001 to 10-100$ spreads. This works very well in the token model because of the existence of market makers who profit from the fees and trading the spread. CLOB market making requires you to have an inventory of the token, and of course, in extreme cases, as is true for all markets, you can be left with a bag in case the token crashes.

Automated market makers popularized by Uniswap allow you to deploy 50-50 pools along the $xy = k$ curve and don’t need you to manage the position actively but suffer from impermanent loss (tl;dr when the token price moves very rapidly compared to when you deploy the position, your Liquidity position undergoes value erosion). Uniswap-v3 fixed this with the CLAMM (Concentrated Liquidity AMM) model by allowing LPs to deploy capital over specific ranges, and you could almost argue that when this tick size approaches the 0.01 level, CLAMMs can virtually emulate an order book.

Taking a step back, though one thing necessary for the CLOB model and the AMM model is fungibility, in simple words, the existence of vast amounts of access to the tokens on both sides of the pair in the absence of which a token is very illiquid. Illiquid tokens are very volatile because even a relatively sizeable order moves the token’s price. and Illiquid markets also give rise to high slippage. You’ve encountered slippage when a buy or a sell order doesn’t get filled at your prices and moves a few ticks or percentages away with your order. More endemic to AMMs because of inefficiency (even CLAMMs), but not entirely absent in CLOB models.

NFTs (or Non-fungible tokens) are inherently meant to be illiquid and are literally (non-fungible). If you were to think of these as tokens and deploy them in either marketplace, you would see any sizeable order move the price of this token.

Current NFT marketplace models employ the order book model almost, wherein if you have an NFT to sell, you list it for a price, and it either gets sold at that price or interested parties offer bids on this which the seller can accept. But due to the inherent illiquid nature of these assets finding buyers/sellers is hard at the price point you might be interested in, and the liquidity is essentially pocketed in various silos, wherein if you were interested in buying multiple NFTs, you’d have to visit every order and buy it separately. This aggregate buying problem has lately been solved by NFT order/marketplace aggregators like GEM and GENIE. But aren’t still convenient for any NFT collectors who possess multiple NFTs of the same collection.

One thing to note is that I have almost treated NFTs akin to simple tokens until now when talking about market dynamics. Still, NFT collections may have an inherent rarity property wherein, for example, in a 10k collection, 8k may be common, 1k may be uncommon, and the top 1000 might possess increasingly rare traits, which also attaches a rarity value to the pricing of NFTs.

The Sudoswap model

Sudoswap is an NFT model tailored to the illiquidity problem of NFT collections. It allows NFTs to be listed in a pool pair, in single-sided pools (either just buy or sell), and the model essentially results in relatively sticky liquidity, which isn’t the usual case with NFTs.

A small dive into the AMM model

The prevalent model of the AMM is the $xy = k$ constant product AMM. While this works in the case of tokens, it still suffers from slippage even when it comes to tokens at volume. While AMMs in NFTs are not entirely new, previous models, i.e. those used by NFTX and NFT20, discretized (not entirely) the curve instead of keeping it continuous. The NFTX and NFT20 model minted an ERC 20 token against your NFT, and you could hold that to experience the gains or loss of the collection that you liked. But given the relatively small and thin liquid books that NFTs usually this also resulted in high slippage. In this case, the NFTs are automatically priced according to the constant product.

Both the examples below assume 10 eth and 10 nfts deposited into the pool

NFTX/NFT20 - Slippage for large orders - https://blog.sudoswap.xyz/deep-dive-1-sudoamm-vs-the-other-amms-they-told-u-not-to-worry-about.html

SudoAMM does a slightly different job, taking the price calculation out of the equation and prices according to preset delta along the possible curves. Below is an example of the delta being set to 10% along an exponential curve.

Sudoswap - Slippage for large orders - https://blog.sudoswap.xyz/deep-dive-1-sudoamm-vs-the-other-amms-they-told-u-not-to-worry-about.html

Because of this external price calculation along a preset exponential curve, the slippage doesn’t worsen.

Also, Sudoswap doesn’t convert the NFTs to erc20 tokens (in some sense, a tokenized NFT, in this case, would allow for concentrated liquidity LP options akin to uniswap v3). Still, the conversion to erc20 always comes at the cost of the UX at some level where the stages of selling or redeeming involve the NFT → ERC20 → NFT cycle, while the Nftx pool initially didn’t allow you to choose the NFT you would like to redeem and was randomized. There’s also the literal cost of wrapping and unwrapping an asset.

The power user playbook

While sellers and buyers can use Sudoswap for single NFT listings, the platform maintains its appeal through low platform fees (0.5%) and cheap transactions achieved through gas optimization.

sudoswap @sudoswap

Replying to @sudoswap

sudoAMM is written from the ground up to be gas-efficient for traders. Trading single NFTs is just as cheap as the most highly-optimized NFT swapping contracts, and when trading NFTs in bulk, sudoAMM can be up to 40% cheaper!

Jul 8, 2022

The natural place where SudoAMM plays an essential role in its use for power users/whales.

One of the NFT whales I follow once revealed his playbook for playing NFT collections. Whenever he liked a collection, he bought about 5-10 from the NFT collection and then sold half at 2x, another half at 3-4x and kept 1 or 2 for the possible moonshot.

It is possible to express this playbook through a simple exponential curve.

The linear curve just sells NFTs in increments of the delta that you specify when you make the pool.

The curves in all unlock

Allow one to DCA in or DCA out of their NFT collection position with a linear curve.
Sell using the Exponential curve and let the curve do the job instead of manually doing it.
Single-sided pools allow Whales to have buy or sell orders for the NFT of their choice, i.e., allowing someone to either sell into their wall or buy from the wall they offer.
Market-making potential on active NFTs setting the fees of your choice to the pool that you deploy while keeping yourself appealing to other alternatives.

A small note on royalty

The past few weeks have been riddled with arguments about the role of royalties in NFTs. Most collections have, since inception, charged royalties on their secondary sales, thus growing NFT collection treasuries and using that for further project development. Admittedly, most NFT collections that have used this haven’t seen much project development. Sudoswap does away with the royalties, and while it is essentially made for collections, the need to not pay royalties also appeals to sellers.

There are a few other routes that collections can take using Sudoswap though

Use some part of the sales, to deploy sell-side pools.
Withhold a small portion of the supply to deploy a pool on Sudoswap and have some fees on top of it, ensuring both skin in the game and earning alongside the growth of the protocol.
While this does not involve Sudoswap, Apecoin, Loot etc, to some extent, have shown how to use tokens to bootstrap a community project fund.
Withhold a small portion of the supply just to sell in the market.

While the general ideas surrounding this don’t come near the royalties revenue that could have been collected, some of these ideas can help ensure a steady stream of revenue for project growth. Given the usual 5% royalty fee seen in Opensea, Sellers might prefer Sudoswap just to circumvent this.

[Note: Royalties are not the only way to support the work of the artists you like]

The Future of Sudoswap

Since its inception, Opensea Users and transactions stand at

Since the inception of Looksrare, Opensea has had the following stats

https://dune.com/hildobby/LooksRare-VS-OpenSea

Sudoswap was launched on the 9th of July and has clocked in about 15 million dollars of volume since then.

https://dune.com/0xRob/sudoamm

Daily volume for Sudoswap since inception

Sudoswap is a relatively budding platform and has achieved more than a few days of the daily volume of around a million dollars (in an NFT bear market). While Looksrare has paralleled Opensea volumes since its inception, it also incentivises its platform’s use and spent about a million dollars a day at the height of its token value to incentivize LPs to list NFTs on its platform. While not at that scale, the Looksrare incentivization epochs are still on.

While some may believe that the below chart of Sudoswap volume growth is because of shilling, and it could be to some extent, I believe that using the product also lets you experience some of the better aspects of the product. Given that NFT markets and NFT collections will just grow in the future, it’s not unnatural to expect Sudoswap to grow and become a more robust alternative to Opensea.

parsec @parsec_finance

Sudoswap market share ATHing on the back of projects minting natively on the protocol

Aug 16, 2022

Sudoswap is in the natural adoption arc of its product while users figure out how to use the product.

There are a lot of possible UX improvements in Sudoswap, which can bring User experience to a whole new level, especially for those who own just 1 NFT. The current experience even asks a user of this kind to deploy an entire pool. The UX could be changed to allow a more straightforward listing while handling these things in the background with some defaults. Other changes include simple call-to-action design changes to the buttons to ensure a smoother buyer/seller experience.

Sudoswap isn’t listed on any aggregators yet, which hinders some of the discovery process for some of the NFTs.

There are many possible improvements to this relatively new delta-based AMM model, including other functions, for example, a simple step function (not sure if the linear model works like this) and more customizations in the pool where sellers might want to choose rarer NFTs to be placed towards the higher end of the curve and sell multiple NFTs at the floor. Other improvements may include a milder version of earlier bonding curve NFT sale models for launching NFT collections.

$XMON today is used as a proxy token for realizing the potential of Sudoswap and has a market cap of 100 million dollars at the time of writing this article, while Lookrare is at 200 million dollars. The last known valuation of Opensea was 13 billion dollars.

While $SUDO does not exist today, the right kind of tokenomics can unlock further platform growth, which does not involve hyper incentivization for sudden use of the platform. I, for one, am excited that a crypto-native NFT marketplace has equipped itself with a product with the right motivations and is now taking on Opensea’s monopoly.

Uniswap as a business?

Mon, 08 Aug 2022 20:16:33 +0000

Temperature check about making $UNI great again.

A short history of uniswap

Initially launched in 2018, from an idea inspired by Vitalik’s post in r/ethereum and a blog post:

Let's run on-chain decentralized exchanges the way we run prediction markets (r/ethereum) reddit.com On Path Independence vitalik.ca

While a proof of concept was first made just before Devcon 3, this lead to an ethereum grant application which got accepted in grants update wave 3. Uniswap v1, as it was called, was launched in 2018.

Hayden Adams 🦄 @haydenzadams

1/🦄 Excited to announce the launch of @UniswapExchange ! It's a protocol for automated exchange of ERC20 tokens on Ethereum. uniswap.io

Nov 2, 2018

Uniswap v2’s overview was first published in march 2020 and was launched just a few months later, just before a period, and probably also a catalyst, called defi summer.

Uni was one of the most thriving decentralized exchange and by September 2020, just before the launch of its token, it had:

From the blog on the launch of the $UNI token: https://uniswap.org/blog/uni

The token launch was almost entirely spurred by the fork of the protocol called sushi which launched the first vampire attack of its kind by incentivizing LPing with the use of $sushi tokens.

The UNI Token

First, let’s take a look at the tokenomics

Token Allocation

The uniswap token launched with a supply of a billion tokens, where 40% was reserved for advisors, investors and the team, while 60% was reserved for the community.

There’s also token inflation which will start at 2% from the 4th year onwards, i.e. 2024 for us.

A quick overview of the 60% given to the community, 15% was airdropped to LPs, users and Socks holders, while 43% was given to the “community treasury”, and the 2% was used to run the only Incentivized liquidity program in Uniswaps existence.

Uniswap V3

Just a year after the launch of Uni v2, Uni v3’s introduction was made and similarly launched in May 2021. By the time v3 was launched around 150 billion dollars of volume had been clocked on Uni v2.

Uni v3 was launched with the idea of Concentrated Liquidity AMM, while the v2 was a $xy = k$ AMM. The CLAMM was, in practice, meant to be more efficient but came at the sacrifice of a more straightforward interface with LP tokens. Liquidity providers had to control the range of their liquidity in a tick-based fashion, similar to what we see in CLOBs. This information capture meant that the LP token could no longer be a simple ERC 20 token and was shifted to an NFT. Also, passive LPing was no longer entirely possible (although the full range option was possible, fee capture would be a few magnitudes lower than those who controlled their LP positions somewhat actively.

Having suffered the issue of forks with Uni v2, Uni v3, for the first two years of its existence, would be under a business license and will convert to a GPL immediately after i.e. uni v3 will be under a GPL license by May 2023.

Was the CLAMM model successful? Since the launch of Univ V3, it has seen a cumulative volume of 640 billion dollars$. Achieving a variety of milestones along its way,

Hayden Adams 🦄 @haydenzadams

🔥 @Uniswap weekly trading volume just passed $10b for the first time!!!

👀 $10b/week is over $0.5 trillion per year h

Apr 20, 2021

Hayden Adams 🦄 @haydenzadams

$1 trillion all-time volume 🤯

I don’t tweet milestones as often these days, but four commas blows my mind.

Never expected Uniswap to grow the way that it has.

Thanks to everyone who has been along for the ride 💜

Uniswap @Uniswap

1/ It’s been one hell of a ride 🚀

As of today, the Uniswap Protocol has passed a lifetime cumulative trading volume of $1 Trillion.

May 24, 2022

Uniswap, on average daily, sees at least 1-1.25 billion dollars of volume daily and also enjoys an almost monopoly status capturing the majority value share in the dex market.

Uniswap share at 71%

Uniswap a Business or a public good?

Since the launch of Uniswap v2, everyone was waiting for the launch of the token. Once the token launched, it debuted as a “valueless governance token” and has maintained that status quo since then.

Despite seeing a trillion dollars in volume, none of the value that could’ve been captured as fees by the protocol has been captured, and, as it stands, uniswap today makes no revenue.

One of the significant contentions for not starting the fee switch on Uniswap pools is that LPs would migrate to other exchanges. While true to a certain extent, what is also true is that Uniswap v3 enjoys its position as the only CLAMM in existence and LPs enjoy capital efficiency just as much. With UNI v3’s business license ending in a year and other concentrated liquidity DEXes launching soon, that status is under question.

Before I talk about the other potential issues and address a recent proposal for this fee switch, let’s do a hypothetical analysis of what it would have been like if uniswap successfully flicked the fee switch on.

The protocol in the past week has seen a volume of around 6.5B $ and has accrued ~7.6M $ in fees to LPs.

Past seven days before august 7th

Assuming that the lowest possible fee is chosen around 1/10th, that is about 760k$, but given that the average weekly volume of uniswap has been around 9-10 billion dollars, that is about 900k - 1M $ per week, or about 46.8 - 52 million dollars of fee accrual, i.e. in simple words revenue accrued to the protocol.

But won’t the LPs migrate? Let’s assume in this instance that turning the fee switch on has an LP value erosion effect, and about half the volume drops, so about 4.5-5B $ volume every week and the fees would amount to 23-25 million dollars annualized.

Even if you assume that about 20% is given to the treasury for protocol growth, about 18-20 million dollars of revenue could be returned to holders of the token, which at current rates would result in about an earning per token of about 0.04$ EPT (only circulated supply considered).

This does not account for any protocol growth and assumes adverse LP effects (which is not necessarily true).

Do LPs in uniswap v3 make money? In a series of blog posts, 0xfbifemboy analyzed the nature of LPs in various scenarios.

In “A Study on Apes: Profits vs Impermanent Losses”, a study of the first 24 hours of LP performance after the launch of apecoin, around 86% of LP positions were in profit, but the majority of them suffered IL loss, the profit of LPs could be attributed to the performance of apecoin. Another aspect is that the median fees earnings were 3.3% of the principal.
In “Impermanent Loss and JIT Liquidity in the Uniswap ETH/USDC 0.3% Pool”, a study of a relatively mature pool ETH/USDC in the 0.3% fee pool. It was found that smaller LPs found it challenging to manage LP positions actively, and as a result, their LP positions drifted out of range (in this case, the LP position suffers from IL and doesn’t accrue any trading fees either). 41% of the LP positions in the pool (when the post was written were not earning any trading fees). About 4.1% of all minted positions were JIT liquidity and it was found that they are highly efficient in generating trading fees.
In “Theory vs Practice: A Deeper Look at ETH/USDC Liquidity Dynamics”, A deeper look at the most liquid pool on ethereum, i.e. the ETH/USDC, across different fee tiers. The concentration of liquidity in the 0.05% fee pool was found to be higher than that in the 0.3%, showing a conscious rational choice by the LPs. While also pointing out that the liquidity in the 0.05% fee pool was dominated by a small number of large players. An analysis of the fee generation revealed that the 0.3% fee pool consistently outperformed fees generated in the 0.05% fee pool, often by double digits percentages.
In “Unraveling a Puzzle: A Per-Wallet Analysis of ETH/USDC Liquidity on Uniswap V3”, An exploration of the ETH/USDC liquidity on a per wallet basis showed that the profits were essentially random. The users with the highest profits were deploying liquidity in narrow positions akin to a limit order, also adding that this was not due to skill but that positions tended to converge to this style of liquidity deployment.
In “Stablecoins on Uniswap V3”, a preliminary study of stable positions in uniswap v3.

TL;DR, LP-ing in uniswap is hard.

Where does Uniswap spend money?

Uniswap has spent money on

Uniswap grants program, which spends about 1.5M $ per quarter. Look here for details of what ideas and avenues have been funded. The retrospective for this program can be found here.
Funding a few liquidity mining programs on arbitrum and optimism
Funding a 1M uni for political defence of defi/ Defi Education Grant.
An ongoing proposal to fund a Uniswap Foundation with 74 million dollars is likely to pass. The proposal can be found here. The proposal also requests about 2.5M Uni for voting purposes.
- The foundation will also subsume the UNI grants program and is asking for 60 million dollars for it.

The planned year 1 for the uniswap foundation

While there have been various reasons to oppose this, the idea of a uniswap foundation somewhat seems antithetical also because protocol management seems to be one of the most significant responsibility of such foundations, and this already happens on uniswap forums through its governance process, which is non-trivial to involve yourself in given the proposal creation and quorum criteria. The raison d’être of this uniswap foundation majorly seems to expand the grants program.

A recent governance proposal

It would be sloppy to miss the recent governance proposal, which suggested a small pilot on this matter.

"Fee Switch" Design Space & Next Steps - Requests for Comment - Uniswap Governance gov.uniswap.org

Something that seems to have been quietly spoken about and being passed.

Some points in the proposal and discussion in this and the follow-up proposals

No information on LP reaction if the fee switch is turned on. Essential to note that an adverse reaction affects the usage of the protocol, i.e. if LPs migrate to other platforms.
- Uniswap v3 enjoys its position as the only CLAMM in the space, and I’m not entirely sure if LPs would prefer the less efficient and more IL-prone AMM model to this anymore.
The fee switch discussion might be premature, and there is no concrete framework on utilization of the fees collected, i.e. how much should be handed out to users, if at all or to who does this accrue?
Potential incentivization of LPs locking liquidity in the protocol.
- LP incentivization has always put the protocol at a loss if fees accrual is considered, and existing examples are yet to be analyzed heavily to support this argument. Sushiswap, one of the popular exchanges, was spending about 48 million dollars annualized at the height of its existence on LP incentivization.
Gamma strategies mention the razor-thin margins in Uni v3 LP-ing and suggest finding other revenue accrual ways instead of putting negative pressure on LPs.
Research on the Economics of AMMs shows (something spoken about in the proposal)
- In a perfectly efficient market, the protocol with the lowest take rate (fraction of fees that goes to the protocol) attracts all liquidity. This is an argument against flipping the fee switch.
- The result is different in a market where a protocol like Uniswap has a “moat”. According to our model, this competitive advantage makes it possible for Uniswap to sustainably introduce a non-zero take rate even when competing with zero take rate competitors.

Legal implications of turning on the fee switch [Unknown]

The current proposal, in its final form under vote in snapshot, includes turning on the switch for the following pools:

DAI-ETH-0.05%
USDT-ETH-0.3%
USDC-ETH-1%

Where do you use protocol revenue?

One contentious issue in the proposal seems to be the issue of “using” this, and in what form will it be used?

Uniswap grants - Currently, Uniswap grants are spent out of the protocol treasury, and revenue could be used to funnel that.
Dividend to UNI holders - Uniswap, if it does, would be one of the first protocols to give back to users without incentivizing the usage of this product, almost every other defi protocol which claims to give back when adjusted for LP incentivization is in a net negative.
While the Uni foundation claims to be an entity for this, every penny spent in Uniswap through governance comes through its treasury (and while the treasury is vast and bigger than even ethereum foundation), this is bottom line fueled growth/spending and surely erodes the balance sheet were you to view uniswap as a business.
Protocol growth

Final Thoughts

While the vote for the above proposal is still ongoing, There seems to be enough pushback inclining the position of a protocol-wide fee switch not possible soon.

Uniswap v3 enjoys a monopoly in the DEX design space today, which is not necessarily true in the future. While the protocol community treasury enjoys about 350 million UNI (about 2.5B dollars at current prices) in its hold, revenue accrual is probably a cornerstone of any successful decentralized finance protocol and uniswap could, if it chose to, become an exemplary defi business.

On the other hand, the community treasury could also be used to guarantee the existence of the protocol in perpetuity, funding various shenanigans along the way and still being a net positive to the ecosystem and enjoying a place as a public good. But would this keep the price of the UNI token afloat? Some proponents of this could argue that the speculation flywheel of crypto would always value UNI at a non-trivial valuation.

While protocol revenue remains a good argument for turning on the fee switch, one of the better arguments for turning on the fee switch is to collect fees from JIT Liquidity.

JIT Liquidity, or just-in-time liquidity, is common in UNI v3. In this instance, a (malicious) actor provides liquidity for a large trade when the trade is registered in the tick/s that the trade is happening and captures the majority of the fees possible. Turning on the fee switch would allow not only to capture part of the fees of this trade (since it’s done chiefly during large trades) it could also be considered to return some of the fees to the LPs to protect them. JIT liquidity also harms the users that have provided liquidity passively and makes IL worse for them. While I’m not aware if the uniswap protocol allows this addition to its code (since it’s not upgradeable), it could also be possible for protocols to add dynamic fee capture (at least on pools where its a common occurrence), which targets JIT liquidity capturing up to 25-50% of the fees for liquidity that’s added during trade execution.

Is Uniswap a business or a public good?

Some Resources

Uniswap v3 volume and fees collected — Dune Dashboard
Dex Metrics — Dune Dashboard
Uniswap v2 vs v3 — Dune Dashboard
Uniswap Governance Forum
- Fee switch space proposal
- Community governance process format
The economics of market makers — Arxiv paper

Verifiable Delay Functions

Fri, 05 Aug 2022 10:13:42 +0000

or How to make things a little slower.

Recently been reading about randomness in ethereum and blockchains generally.

Randomness on blockchains doesn’t exist in ideal form and is hard to find. The use of randomness has various applications like raffles, lotteries, or perhaps an auction model where the initial minimum bid is randomly chosen from a list. On a protocol level, the protocol can also use this randomness to achieve privacy or select a random block producer in the case of proof of stake consensus.

The closest we come to is using chainlink VRFs which, when provided with a seed, generate a random number published on-chain with proof and can be verified using the oracle’s public key and application seed. VRFs have been used in crypto in various applications and at the protocol level in Cardano (Ouroboros) and Polkadot.

Another way to achieve randomness is using Verifiable Delay functions, proposed by Dan Boneh, Joseph Bonneau, Benedikt Bunz and Ben Fisch.

Consider the scenario where you have to run a lottery. While the usual way would be to give everyone tickets and draw from a random set of numbers, we want to run a verifiable lottery so that no one should be able to manipulate the numbers drawn/generated. Randomness beacons are of help here. What is a randomness beacon? Randomness beacons are public sources of randomness which emit random values at constant times. The generated values must be such that they can’t be manipulated and be predictable. How does one go about making them? One way of doing so is to “apply an extractor function to a public entropy source”, basically using a function which can convert highly unpredictable signals into random numbers. The randomness comes because these signals themselves are hard to predict.

For example, think of using the values of 100 stock prices. You can use the closing prices and then hash them to get a random value. A motivated adversary (with enough funds) could manipulate the stock prices of a few stocks, though (let’s assume $k$). In this scenario, the adversary would read the prices of the stocks it cannot control ($100 - k$) and then quickly simulate $2^k$ scenarios based on the outcomes that it can change (i.e. manipulate the stock of the price upwards or downwards). Then this would generate some number which is more favourable to the adversary.

Some outlandish suggestions to improve this could, of course, involve applying extractor functions to the weather or more natural phenomena which are beyond human control. Another way of doing this would be to add a delay function after extraction. In this case, let’s say the market closes at 3:30 pm and adding the delay function makes it so that the random value is generated at 4:30, and the prices are finalized. The adversary would not be able to manipulate or simulate the outcomes of any such strategy.

Simply stopping the clock for an hour and using the extractor function on the prices is not sensible since the adversary will also be able to do it. Hence the nature of the delay function should be so that any adversary, regardless of having access to massive parallel computation, still need to spend at least $T$ time (an hour in our example) to arrive at anything close to what the delay function does.

Verifiable delay functions are cryptographic primitives that have been formalized to act as delay functions with additional properties.

Now let’s take a formal look at this.

VDFs have the following advantages: (from Vitalik’s post on VDFs)

Relative to RANDAO and similar schemes, they cannot be manipulated.
Relative to BLS threshold signatures and similar VRFs (verifiable random functions), they do not depend on any specific fraction of nodes to be online and do not require a complicated setup procedure.

Verifiable Delay Functions (VDFs)

A VDF consists of 3 sets of algorithms.

1] Setup $(\lambda, T)$ - This function takes security parameter $\lambda$, and delay parameter $T$ and outputs public parameters $pp = (ek, vk)$, where $ek$ stands for the evaluation key and $vk$ stands for the verification key. The setup function also fixes the range and domain of the VDF. The delay parameter $T$ acts as the time-bound.

2] Eval $(pp, x)$ - This function takes an element $x$ in the domain, outputs a $y$ in the range, and produces a proof $\pi$.

3] Verify $(pp, x, y, \pi)$ - Given $x$, $y$ and proof $\pi$, it verifies that $y$ is the correct output of $x$.

Another thing to note about the setup stage is that it might need an initial trusted setup.

A VDF should satisfy the following properties:

1] Uniqueness: if $\text{verify}(pp, x, y, \pi) = \text{verify}(pp, x, y', \pi)$, then $y = y'$

2] Sequentiality: if $A$ is an algorithm which runs $(A, x) < T$ (the delay parameter), then it can’t distinguish the output from some random number. One of the ways to do this is to have an iterative application of function where in you apply $f(f(f(f \ldots f(x))))$ $t$ times here. Regardless of parallel computation, you would have to run the function $t$ times to arrive at the result, but in this case, the proof can be slower to generate (and this is also one of the bottlenecks of creating efficient VDFs). This can also be called $t$-sequentiality.

3] Efficiently verifiable: Once the output is generated, it should be easy and quick to check that it is the correct output.

Simply speaking, what we require from VDFs is:

The function should be slow to evaluate for both the honest agents and motivated adversaries who have a lot of parallel compute or specialized hardware
The function should be easily verifiable.

Slide from Joseph Bonneaus VDF Talk at Protocol Labs

Construction of VDFs

A theoretical model presented in the paper

As spoken about in the sequentiality section, one way would be to apply the function $t$ times, and this can be accomplished through a Hashchain (which is just hashing the input $x$) and then hashing it again up to $T$ times to get $Y$.

Hashchain with Verifiable Computation

The way to make a Hashchain verifiable would be to combine it with SNARK or STARKs to generate the proof. The issue with this is that $\pi$ is incredibly slow to calculate.

One immediate improvement to this procedure is to break down the Hashchain and generate the proof iteratively.

IVC: Incrementally Verifiable Computation

where in you apply the hash function and generate the proof for a few hash applications and then use it to generate the proof after that, so generating the proof while you create the output.

A few optimisations can be made to make it snark optimized, like choosing SNARK-friendly hash functions or replacing the $H$ with a permutation that is slow in the forward direction but fast in the reverse direction.

While treatment of the permutation functions considered is beyond the scope of this blog, permutation polynomials usage for a VDF is still an open question.

Slide from talk about VDFs at IACR

You can take a look at this talk for more.

Simple and Efficient VDFs

The papers, “Simple Verifiable Delay Functions” by Pietrzak et al. and Efficient Verifiable Delay Functions by Wesolowski et al. show possible ways VDFs can be constructed.

The only matter they differ in is the way the proof is generated.

(You will need some knowledge of group theory to follow along after this)

An Algebraic Construction

Slide from talk by Dan Boneh at Eth Foundation

So in this construction, you assume you have a finite cyclic group of unknown order.

The Setup of VDF will be the unknown order group and the hash function, which maps the input $X$ into the group.

The Evaluation will be $y = H(x)^{2^T}$, which in the case of finite groups will belong to $G$ itself, as seen above.

Example for Finite Groups of Unknown orders

It’s very important that you don’t know the order of the group because if you know the order of the group, let’s say $N$, the evaluation function reduces to a much simpler:

$$ y = H(x)^{2^T \bmod N} $$

which is trivial to solve for smaller order groups.

One thing to note here is that to assure security; you probably need to choose a very large $n$ (around 4000 bits) to make it secure enough that the adversary can’t crack.

The proof for this, called the proof of correct exponentiation, can be seen in the papers and has been arrived at in different ways.

Efficient Verifiable Delay Functions

The motivation for the construction is from Time-lock puzzles by Rivest, Shamir and Wagner, where they use an RSA group to lock the time-lock puzzle for a period of $T$ seconds and arrive at a much similar formula.

The interactive argument/proof for this would look like this:

From Benjamin Wesolowski’s talk at Eurocrypt 2019

Interactive proofs, as suggested by their name, require interaction between the prover (Alice) and the verifier (Bob), but this proof gives you an idea of how it can be proved and then converted to a non-interactive proof with the help of the Fiat-Shamir Heuristic.

Assume the function, $x^{2^{10}}$

Walking through a small example for convenience, this would look like, as mentioned above, $2^{10} = 1024$,

Bob, let’s say, chooses a prime 13
Alice finds the $q$ and $r$ given $e = 13$, which turns out to be $1024 = 78 \times 13 + 10$, so you arrive at $q = 78$ and $r = 10$
Alice sends back proof $\pi = x^q$, so you end at $x^{78}$
Bob computes the remainder, $r = 10$ using $e$
- Accept if $\pi^e \cdot x^r = y$
- In our case, this would be $(x^{78})^{13} \cdot x^{10} = y$
  - This results in $x^{1014} \cdot x^{10} = x^{1024}$, which we started with.

The non-interactive version of the proof iteratively takes the next prime number, which satisfies this proof.

In the case of Simple Verifiable Delay Functions by Pietrzak, the proof is in the form of an interactive iterative protocol and is then similarly converted to its non-interactive form.

NOTE: Something I’ve not covered in this blog and is just as important is the complexity analysis of these protocols (efficiency/ how fast they are), and that’s an important criterion in selecting one construction over the other.

Application of VDFs

VDFs can be used in

Leader Selection (in Resource Efficient Blockchains) - While a number of blockchains already use VRFs and other techniques to select leaders. VDFs can also select new leaders at regular intervals using a randomness beacon.
Lotteries
Randomness Beacons - VDFs can be used to make public sources of entropy (as discussed at the start), like stock market prices, a randomness beacon.
Computational Timestamps
VDFs can also be used to provide transaction randomization to in fully private protocols, as told in A note on Privacy in Constant Function Market Makers (which I will cover next).

VDFs are relatively very new cryptographic primitives which have immense application in generating randomness with few assumptions and can also boost privacy and other mechanisms in the crypto space.

The open questions for VDFs are

to generate a Quantum Resistant VDF.
Are there other groups of unknown order?

Some Resources

Verifiable Delay Functions — Original Paper
Efficient Verifiable Delay Functions — Wesolowski et al
Simple Verifiable Delay Functions — Pietrzak et al
A Survey of Two Verifiable Delay Functions — a survey paper of the above two methods
Introduction to VDFs — Talk by Joseph Bonneau at Protocol Labs
Verifiable Delay Functions — Talk by Ben Fisch at IACR
Verifiable Delay Functions — Talk by Ben Fisch at BPASE 18, Stanford Cyber Initiative
Verifiable Delay Functions — Talk by Dan Boneh at Ethereum Foundation
VDF Research — Maintained by the VDF alliance; to look at work that has happened in various directions since VDFs came out.

The next million users

Fri, 29 Apr 2022 07:32:46 +0000

If you think about it we’ve come a long way since defi summer of 2020, almost had an entire cycle at this point, onboarded a variety of users and through NFTs even are reaching the broader culture base of humanity. We are now experiencing a new wave of defi innovation and experiments within NFTs, trying to scale and adjust our products and tools to onboard users to the future of finance and the metaverse.

The other day, A friend of mine was just talking about inflation and how the best banks in his country gave around 1-2% in returns on savings. And when he added the inflation, the real rate was well in the negative. I offered him a few alternatives as to how he could buy index funds or ETFs and DCA into for a year and then maybe the stock market will have done its correction by then. As a final option, I also suggested crypto, the likes of luna providing around 20% and even the safest choices giving around 6-8% which is a few multiples that you’d find in CeFi institutions at the moment. So then he asked me, “Where should I get started with crypto?”

So where does someone who has no idea what wallets are, what a CeX is or what Defi is get started in crypto?

The usual steps let’s assume you are interested in returns on a CEX would involve:

1] Logging onto any exchange available in your country

2] Adding KYC (if mandatory)

3] Connecting Bank Account

4] Depositing Money and getting this changed into your choice of stables (at this point we have so many, that getting confused about this is natural, but since most exchanges seem to prefer USDT pairs for volume more we use that)

5] Checking the terms on the returns on the exchange, seeing if flexible or not and then depositing in it. [In the case of Binance there are a few direct products which they deploy onto venus, or some other defi they’ve made available, In the case of FTX you’d have to look at lending and borrowing rates and take a call]

Going a step ahead, What are the steps for someone interested in Defi?

1] The First step would be to pick the desired product that you might be interested in

2] Check the chain that the product is on and if the chain allows direct withdrawals from CEX

3] Choose a wallet [anyone who’s been in crypto long enough advising a newcomer wouldn’t fail to mention the advantages of using a cold wallet over a hot wallet]

What wallets to use? While most EVM compatible chains metamask would be the choice of product, Solana would require phantom, Polkadot something else (but moon river being EVM compatible allows usage of MetaMask), Tezos would require Temple or Kukai. (Note: Yes I’m aware that most chains don’t have any notion of defi product so they shouldn’t probably be considered)

Then comes a point where you explain to them how any wallet they make has 12 phrases that they might need to store somewhere secure, Not tell this to anyone (of course), loss of this seed phrase would also result in loss of access to funds.

Note: At this point, a question worth asking definitely is, for people who say “not your keys not your funds”, we definitely haven’t made managing keys any better in the last 5 years and it’s gotten so complex that even the most veteran of crypto users simply find it as a cumbersome pain-point they choose to overlook because of the money at stake.

4] So now comes the stage where you transfer funds from the CEX to your wallet, please advise them to take caution and double-check or triple-check the address lest they send the money to a wrong address resulting in loss of funds again. [There now seem to be more ways to lose money in crypto than making them.]

5] Check if funds are received.

6] Go to choice of website, Choose to connect network, Convenient if ethereum.

If not ethereum, educate them about RPCs, make them go to chainlist.org, and add the RPC of the desired target network.

7] Once on the website finally deposit your money, but again have token approval, transactions to deposit money also inform them about another transaction for any removal.

What if the CEX doesn’t allow direct withdrawal to the network?

1] First deposit to any of the networks (while a lot of people might choose ethereum here, most normal first-time users are simply “priced out of ethereum”).

2] Choose one of the several bridges (honestly we just might need a bridge aggregator at this point).

3] Deposit from the bridge to the target chain.

Now you also have to experience the hell of the funds being withdrawn from one wallet and having to wait on the other end where the funds haven’t appeared. You experience a few minutes of hell if bridging large amounts of money for the time the money is being bridged and doesn’t appear on either end.

What about exit? Do all the steps done until now in reverse.

There are a few direct onboarding services which avail you the option of directly getting fiat on-chain but haven’t picked up enough steam (or are unavailable in most countries).

UX in DeFi

While we spend our time forging the next set of products, the next set of “ponzis” as we like to call all of them, the next wave of defi, I think we at the same time aren’t paying enough attention to making UX easier.

The next 1000 wave of users, the next 100 thousand, the next million, the next billion users for Defi will only come when defi is accessible.

So do we have accessible defi today? Can you recommend a product which is painlessly easy to access? I don’t think so.

I think seamless onboarding on-chain should almost be classified as a public good.

What alternative world would you see if this doesn’t happen? Something I’ve been worried about which might not necessarily be a bad thing for most but doesn’t exactly posit the ethos of defi is that if good tools of access are absent then CEXes will start integrating access to these products through their frontend become the one-stop-shop to defi access and will onboard the next wave of users and it’ll only be the veterans and the fogies of crypto left being onchain (also the CEX would pocket a slight fee).

Decentralization at its most basic level is about a choice, it’s not about having to use the CeX that is bad, it’s the inability to opt for an alternative that makes it bad.

Forget about Decentralization, Anyone who creates a product has to make sure that all parts leading up to the usage of the product are easy and well oiled. Otherwise, history has then and again shown that there will be some competitor, someone who creates a product offering “this well oiled seamless usage” as a product. “Your margin (of inefficiency) is my opportunity”.

Wallet Safety is one of the sectors paid the least attention to within the user design space of crypto. Better wallets have certainly led to better designs maybe, but not more secure or recoverable ones. We like to make fun of every bored ape owner that has lost his ape, but perhaps the BYAC owner class represents the most normie class of users in crypto and those are the users we are trying to onboard next.

While we might have a long list and threads about wallet safety, maintenance of hot wallets, securing seed phrases by not typing into random websites, not going on malicious websites, we do not question the design of the way we operate with wallets today and that might be crucial moving forward. The easiest abstraction here which would even appeal to regulators is to outlaw defi usage on-chain and make CEXes the only way to use the products thereby also securing KYC requirements. NFT marketplaces seem to be more and more commonly launched by CEXes these days exactly because it is easy for them to cater to users and NFTs seem to have found a PMF with normies more than defi has (perhaps NFT resonates with most people because of its mimetic nature). We are a long way from good custodial solutions.

I hope we see more experiments in social recovery wallets or even custodial wallets with user control (which is less centralized than the CEX owning the wallet). Fiat on/off-boarding and Wallets have been ignored for long enough in our conversations, but perhaps they are the crucial legos to unlock the next billion users of our products.

Meta - Ramblings of an inexperienced player

Mon, 28 Feb 2022 20:27:48 +0000

Writing down some notes of what I’ve seen in the market for the last two years.

I’m not quite sure where the market is going, it looks pretty fatigued. When I came back to trading/investing in crypto in summer 2020, I was under the assumption that this cycle would probably end around us topping at 5 trillion dollars. This article is perhaps just a recollection and retrospection with some learnings of what I saw since then with some Dota analogies (bear with me through those).

Note: Someone told me that this work reminds them of Cobie ser’s, Trading the Metagame, So you can probably skip this if you have read that, This is probably just an extensive exploration of that idea dating back to defi summer 2020.

I’m a DOTA 2 player (Defense of the Ancients), and it’s viciously addictive having spent 3000 hours playing the game and even then still being engaged by it, I often wonder what makes me go back to the game? Most modern RPGs are played around for 100 hours to complete the main storyline while 200-300 hours for covering everything. DOTA 2 is an MMORPG (pronounced morph) and something that keeps getting changed due to patches, the heroes are balanced/the map is changed/the items are updated, and that keeps it very interesting.

There is this dialogue in the movie, The Prestige by Alfred Borden,

And while that is true about magic tricks, that is true about games as well, “figured out” games simply aren’t interesting, and every patch in dota brings with it new things to figure and new interactions to explore and new things to learn. There’s probably something about stochastic games, partially observable games which keep humans engaged (since we are pattern recognition machines constantly trying to figure stuff out).

So what happens when a patch is figured out, what does it look like in a game, Some heroes are found to be more broken than others they are simply better given of the in-game economy and the items available (in their current form in this current state of the patch). And this is when a meta develops.

The meta is almost always created by the players. They try several builds and items and some strategies eventually tend to work much better than the other ones. Even with relatively “balanced” patches, some things become more popular than others, even though there might be other broken things (due to influential people, pro players in the context of a game). Developers then nerf certain things that are too broken and another cycle of finding the next meta commences, also done so that the game doesn’t become too stale. That probably is enough of a digression from the point of this article. But the existence of a big action space, by that I mean, there are too many things to try out always means that even though there might be some things popular in a meta, given that the popular stuff gets nerfed, there might still be things in the system that can be exploited or found to have an edge over others.

But what does the meta of a video game have to do with crypto? Abstract models translate to other places in the wild too. Crypto being the hyper-financialized paradigm of tokens and projects that it is, is very much like the partially observable, stochastic games that are played in dota, league of legends or other MMORPGs. The meta is the tokens that are currently going wild, the projects that have the maximum attention devoted to them because financial regimes seem to be driven by intrinsic attention economies. The thing about attention economies is nothing can stay in its place forever and the meta dissolves (in this case this certain class of projects die) and then either it’s on to another class of projects or everything is just silent for a while. Why is knowing any of this important to you? Knowing about the existence of the metagame perhaps helps you discover them or recognize the one you are in or discovering some common trends in the different metas could help you know if the shitcoin that you currently have has something unique in it to drive its own meta. After all, you play to win.

While when the run in summer 2020 began, crypto was but a 250 billion dollars worth asset class and I was initially even sceptical of it ever reaching the trillion-dollar mark (because that for me was the hallmark of a mature asset class) the highest it ever went in the 2017 run was 850B before collapsing to 150 billion at its lowest. It has 10xed in 2 years achieving an all-time high of around 2.5 trillion dollars and currently sits at 1.7 trillion. But I think regardless of that what is quite true (and might stay true unless we go sub trillion as a market), An asset class worth a few trillion dollars is liquid enough to fund its own microcycles and anyone who understands this is positioning themselves for some gains (if not a lot).

Token Metas

I will first cycle through token projects, and then later talk a little bit about NFT cycles (there haven’t been many). Also, some of the chronology might be misplaced or just simply wrong but I’ve tried to recapture the major ones that I recall.

Nothing taught me about information asymmetry, which could be something as simple as knowing to use a product or putting different things, as much as defi summer 2020. The early projects were yam, yearn, harvest and other projects which at this point were called defi even though all they had was TVL based token emission slowly getting diluted through pool 2s and had insane APYs which hovered from 6 digits to 10 digits. Anything less than 5 wasn’t even considered. Most people were still trying to figure out the process of depositing tokens → getting LP → depositing LP share → claim rewards, which wasn’t clear early on or were too scared because the APY looked too good to be true. I think a dominant signature of different metas in crypto and I will mention this latter too, is that you can identify the ending stages i.e the meta attaining critical mass by the number of rampant forks that it spawns. Kimchi, Based Finance and other names I can’t even recall were forks attaining 9 figs worth TVL less than a day into the launch and losing it just as quick.

Simultaneously around the end of the yam cycle which ended with a smart contract bug (which would’ve or did, I don’t recall, led/lead to loss of funds), developed the rebase token project meta. You could recognize this by the number of people talking about how the “elastic supply” of tokens led to better tokenomics. Everyone suddenly knew what elastic supply meant and what number $AMPL or some random rebase token would have to be at to rebase positively or negatively. Something that is very true, particularly about financial games is that (only?) being early helps and that the later you are to the game, the more likely you are getting dumped on, so if you recognize that you are late to this meta you are probably better off avoiding it, because you probably don’t know what stage you are in and could probably end up losing a lot if you are late.

In the shitcoin projects that I saw something that became popular for a while after this was deflationary tokens which rebased, every transfer had a penalty but holding the token availed you rebases and unlike the earlier rebase tokens these were strictly positive rebases. This did not last for very long since it led to quick inflation of the supply and the bigger participants profited very quickly and moved to book their profits.

Soon after the rebase coins, late 2020 saw the rise of algorithmic stable coins. This was the introduction of game theory (somewhat so) to tokenomic design and has been tried in a variety of ways since, but early on was about how long the stability of the ecosystem could be kept propped up through bond sales and the emission of the stable coins to bring the value of the coin closer to 1$. Empty Set Dollar (ESD), Digital Set Dollar(DSD) are just a few names that come to my mind when thinking of these, both of them trade at a cent or so today, so while it was an interesting experiment per se, it’s clear that they didn’t survive.

Q1 2021 was a period during which almost every token was going up only, and you didn’t have to exactly be right about the meta to make money in this period. Tokens were up 10-20xes in 30-45 days and the saying “you don’t have to be right in a bull market” comes to my mind quickly. Around early q2, dog coins and other meme-able name coins (like cummies, safe moon etc) started getting more popular and while you could call this a meta or an overheating market (since you would see no value in these tokens), its true that this part was one of the most degen parts of the cycle.

While late Q2- early Q3 2022, were somewhat muted something that came after this was the L1 cycle. Where all L1s were suddenly pumping, you could say that this was a confluence of several things and while it would be untrue to say that these alternative L1s didn’t work before then, it’s coincidental that everything came together for this to happen. A popular tweet by the fox would probably sum up Q3 2022,

Hsaka @HsakaTrades

SoLunAvax

Hsaka @HsakaTrades

SoLuna

Aug 18, 2021

Something curious about L1 cycles was that they were first token pumps and then ecosystem pumps, where the value of the token went up and then the projects on-chain went up (somewhat consecutively instead of in sync), and these in-ecosystem pumps had their own micro cycles which lasted from days to weeks. This also went onto other L1s not just the ones listed above, but the fact that those projects now inhabit the top 20 in the space speaks about their success more than others.

Late Q3, saw the rise of Olympusdao. A POL service that was launched in March 2021, which took some time to actually get stable, was countercyclical for the entire dip in Q2-Q3 because of the coin being solely backed by USD reserves. The thing that really made Olympus popular was the narrative that surrounded it and (3,3) which will probably be remembered as one of the hallmark memes/narratives to come out of this cycle. Something that quickly became obvious alongside the rise of Ohm was that in a multi-chain world, the meta would also become multi-chain and forks wouldn’t be restricted to the same chain. The rise of Time, Rome, Snowbank, Spartacus, Invictus are proof of this and something to keep in mind for the future, something that was fascinating to me about multi-chain forks was the fact that it did not fractionalize liquidity from the main project. Will seamless multichain interop change this? Only time will tell.

For a moment or two in late 2021, early 2022, projects with ve-conomics especially those that involved themselves in the curve wars started pumping. It did not last long enough for me to credit it and call it a “meta”. But something that has definitely been true for me over the last two years has been the following

If crypto is a game, tokenomics is the meta - @cobie

Most crypto micro-cycles have been driven by the tokenomics of the project, which get exploited for maximal gains and quickly fall down as well. Do projects in these metas contribute something to the long term health of the ecosystem? As I write this article, ohm forks (which were the last meta) are down 90-95% since their market top but it’s undeniable that they pioneered the liquidity as a service idea and continue to through Olympus pro. Also, the game-theoretic algorithmic stablecoins were an important experiment which probably couldn’t have been done were it not for the existence of crypto, and finally, something true about all these microcycles is that narratives and communities in projects matter.

state @statelayer

kinda annoyed at defi being a never-ending game of finding new ways to make emissions appear as revenue. like, sure, i'll play the game and trade them, but it's a bit tiring

archived 2022, tweet since deleted

Most innovation in defi seems to revolve around making the tokenomics more appealing to the holders, and while that may be an interesting problem to solve. It won’t increase the surface area of participation that we all aim or hope to see.

NFT Metas

If NFTs are a game, style is the meta.

NFTs are relatively a much more nascent market compared to token projects but may at least provide some solace to the holder because they can perhaps say that they were in it for the art (if the art is good). I will keep this part short.

While crypto punks were extremely popular they never quite garnered heavy attention until late 2021. One of the first NFT micro-cycles was developed elsewhere in the project called Hashmasks. Hashmasks started the bonding curve meta, not quite bonding curve as much as selling blocks of the supply in increasing order until the supply was over (which was tried for at least 4 months by a variety of projects). Something that was quickly evident with NFT projects was that community engagement matters. Hashmasks were the heartthrob of crypto Twitter, while everyone figured out hidden meanings in the artwork, a variety of servers spawned which catered to different trends within the artwork. Hashmasks also was one of the first projects to have more volume in sales than crypto punks in a given day. Hashmasks today does less than double digits volume in a day (if any at all).

NFTs were quick to die in the event of the May 2021 meltdown.

Since then, in late q2, NFTs saw the PFP meta which was led by cryptopunks and then extended to byac, doodles, cool cats and various other PFP projects.

Something that was very popular and in retrospection definitely just a meta within the NFT space was the generative art NFT series. Q3 2021 saw the rise of a variety of sub art projects from Artblocks. Fidenzas, Singularities, Subscapes, Archetypes were just a few names that I can recall from the 15 or projects that were existent then which commanded high valuation up to as much as a few thousand eth for a Fidenza sale. Generative art was hailed because of a variety of things, and you can only question the veracity of those claims now that they seem to have had a similar fate to that of hashmasks. Why did generative art fail? I don’t think it was as much as generative art that failed, as much as it was the reason of the interest in these projects. The projects are not any less aesthetic today than they were at the top. A primary reason of interest for Artblocks was the collections had about 1024 supply and artificial scarcity (low supply is good) remains one of the best ways of pumping the value of an NFT supply. Artblocks kept releasing about 3 new art series every week since q3 2021 and you could say that ironically supply fungibility was why the gen-art meta came to an end.

Something that started crowning its head at the end of the gen-art cycle were the Loot projects which in its most idealistic sense could be thought of as seeds for games in the future/ or art in the case of the n-project. Loot was hailed for a variety of reasons and perhaps development for the game still happens today ( i am unaware, although i did see a project launch visualization for loot recently). Loot also saw the rise of the NFT-token meta where every loot holder was airdropped tokens (AGLD) which would be used in the game economy. This idea has since then been adopted to a variety of projects, and every NFT project plans to launch a token for its supply or is at least actively considering it. Something to wonder about when it comes to this is if it’s a good idea to 1] fractionalize liquidity from the NFT volume to fund a liquid token? 2] Are liquid tokens necessary for something NFTs? And while these questions might have been ill-formed I hope you get the idea.

As of the time of this writing, we recently just saw the anime-pfp microcycle, which didn’t last as long as I expected it to. but NFTs have definitely seemed to have had their own economy insulated from that of the wider crypto market perhaps due to their mainstream appeal.

I might have missed mentioning a variety of projects or cycles in my article but I hope i have mentioned enough for you to understand the broad strokes of this cyclical market that we thrive in. One thing that I definitely missed from the top of my head is the trading card NFT cycles and classifying projects like Aurory, magic and others to the metas that I have mentioned.

A small crossover that was seen and has since then developed and will probably keep getting more common until it blurs all the lines is the Play to earn ecosystem. The market resurrection in q2 2021, was led by Axie Infinity, a play to earn NFT game where you had to buy the Axies to play the game and were rewarded. A lot of projects have since then tried to pioneer Play to earn and have been successful to various measures, but to date, no game seems to have had the same success that you would find from a game from a traditional gaming studio. There was also a micro P2E cycle which was sparked by $jewel, a token for the economy of the P2E game, DefiKingdoms. Something that was fascinating about DFK was the fact that it had bought all aspects of crypto into the game, was an intersection of various concepts and was perhaps one of the only projects which led the underlying L1 to be used more than it previously was.

I wrote this article to keep reminding myself of the following

https://twitter.com/hsakatrades

That recognizing that the market funds micro-cycles allows you to keep playing the game even when the wider market might be consolidating but also recognizing meanwhile that fatigued markets do take breaks and these microcycles only get rarer in these markets (this is when you start planning to take a break).

It’s frustratingly simple to convince yourself that you could have seen all the signs of the meta developing had you done something differently because it is in retrospection. You can keep reminding yourself of the signs that you saw in the previous microcycles and keep applying them to find new ones, but something that has to be kept in mind is that Metas don’t always develop quite the same ways, some achieve critical mass very quickly (like algorithmic stable coins) while others have to survive and the narratives around it have to develop for them to achieve it. Knowing what will happen with the shitcoin you are invested in is almost next to impossible, and perhaps that is why I end with a note to myself, “don’t get married to your bags.”

Something that is no longer true as it once was is the ability to keep in touch with everything in the market today. I could name almost 90% of the hot projects in early defi summer, actually, almost all of them save a few. Today I barely know a few real projects forget hot ones. It’s hard to keep track of a space that has about 13k projects (consider even the first 1500 for brevity). But as Jeff Bezos once said, “Your margin is my opportunity”, something that is true for someone new entering the space is that “your lack of attention in my opportunity” (quite doesn’t roll off the tongue as well). The game gets harder when there are more participants because newer cycles might not even get the time to fully develop before they are dominated or found out by the market. Information asymmetry keeps growing and the meta gets hidden from public spaces eventually becoming non-existent to the public.

Its been an interesting mental exercise to consolidate these ideas and I hope they’ve been of some interest to you as well.

A list of open problems in DeFi

Sat, 29 Jan 2022 06:16:07 +0000

I think that having an open list of problems is a good reminder of progress for a field, nascent or old, research problems or practical ones. These also provide a benchmark to look towards and glance upon to take stock of the progress it has done over the years.

While there is no doubt that yield stacking is impressive and that we can create extensive levels of leverage by stacking things on top of one another. The stick of measure will be solving problems that either extend the ethos of decentralization to every real-world product or rectify the errors that current systems possess or fill the gaps to make the system seamlessly functional, all of this leads to more adoption or reduces the friction/inefficiencies in the ecosystem.

Following is an incomprehensive list of defi problems, I will take stock of these problems in 12 months.

Automated risk scoring of lending borrowing pools -> Increasingly important problem
- Possible Solution: Risk assessment without historic data is really hard because it could be so that a pool of users with a good credit history will always pay the loan back. “Credit score” in tradfi.
  - One alternative way of looking at the problem would be, looking at a function for calculating the probability of default given the pool of assets you have.
  - Thinking of risk in the extreme terms of everyone defaulting although useful is an edge case, every time someone defaults (since these are collateralized pools) there’ll be a liquidation. So this means that there is an existence of a liquidation price, so one way to view this problem (even without considering the users at all) given that I borrow X amount at Y price, What is the chance that price hits my liquidation price Z (in this case the user will have defaulted and will get liquidated).
  - How do you calculate/model the price action?
    - One naive and simple metric would be a weighted average(weight by asset in a pool of n assets) of historical/implied volatility (because in a highly volatile asset it’s likely that the price reaches your liquidation price) and you can do it on a basis of a monthly epoch (now this is a very sanitized way of thinking about pool risk because an asset can improve its order book/action price becoming less volatile)
    - The added advantage of doing a weighted average across a function (IV/historical volatility) is you can just add another variable modelled through ML or some other way and get a more refined answer
  - The assumption of modelling the problem this way is that a borrower might borrow from a pool with less risky/volatile assets.
Managing Risk for lenders and distributing risk/ Undercollateralized Loans
- Defi Lending is not in a place where it can be used by traditional or real word entities because of over-collateralization
- Most real-world loans are undercollateralized
- This under collateralization ensures that the lender establishes the borrower’s ability of repayment
- Tradfi is plagued by NPAs but still ultimately fall back to some sort of credit score establishment [Spectral finance solving this, but still an open problem].
- But still, most credit score methods would rely on onchain history for credit establishment, we are moving towards privacy-centric defi is this approach extendable to that idea? [Homomorphic encryption could provide a solution]
Uniswap v3 is one of the most used AMMs, with over a few billion dollars in it, What is the optimal strategy for lending for both passive and active liquidity providers? How can you formulate and model the problem?
- Most vault managers have failed and only charm finance appears to have some stronghold over this having consistently beaten full range v2 performance

This tweet is no longer available.

x.com/i/status/1483989467744591873

The tokenomics problem [Millenium problem of defi]
- A survey of different tokenomics designs and what they have accomplished for their respective projects
- is veToken the optimal token design for most protocols?
- What are some good boilerplate tokenomics that a project could look into?
- Solving the Liquidity dilemma for most protocols, is POL the final answer for this?
What does a privacy-preserving CFMM look like, and what privacy guarantees can it make? From the paper A note on privacy-preserving constant function market makers
- Answered to an extent in Differential Privacy in Constant Function Market makers
Private Lending
- Lending practices today involve an almost on-chain registry, how do you make this more private
  - Imagine knowing what every user borrowed and paid in the real world
Derivatives
- Today’s on-chain options are not a success yet [The metric here would be compared to volume compared to centralized counterparts]
- Primitive is making some headway with this
Compiler → I give you a payoff and you “compile” it by stringing together other primitives
- One way to think of a compiler is, you are interested in a certain payoff from the assets you have so you have a compiler that auto-selects and tries to model the desired payoff from the instruments/primitives available on-chain, “auto yield stacker”
Gas derivatives
Cross Margining Systems, from The Defi Prime Broker.
Soulbound Governance & Proof of excellence, Inspired by Vitalik’s Blog Soulbound, Hard Problems in Cryptocurrency: Five years later
- Governance today especially in defi is misaligned with the ideas that push forward long term development and choices, Is there a way to recognize and give non-transferrable say in matters concerning the protocol.
- Recognizing a way of who should be soul bounded also solves some airdrop dilemmas which haven’t had much progress since 2019

Hard Problems in Cryptocurrency: Five Years Later vitalik.eth.limo

Talking about Economic problems from the above blog post, we’ve come a long way in some cases since 2019. Stable value assets have seen a lot of exploration and development, looking forward. Stablecoin health dilemma [Centralized ↔ Usage ↔ Mechanism]
- Is it dependent on incentivization for maintaining peg, how healthy is this?
Decentralized Public Goods Incentivization
- While there are experiments with retroactive public goods incentivization and GitCoin also helps in this area, there is a lot of exploration in this space left
Reputation Systems
- Why is this a defi problem? A good decentralized reputation system can do away with a lot of requirements that would need a private defi ecosystem to thrive
Oracle Systems which give access to more varieties of data
- Comments from SoK: Oracles from the Ground Truth to Market Manipulation
  - Two conditions seem necessary for securing oracle systems: the market capitalization of the token stays material and the token is evenly distributed.
  - Oracle systems with on-chain modules are expensive to run on public blockchains like Ethereum, which prices out certain use-cases that consume a lot of oracle data but do not generate a proportional amount of revenue (e.g., Weather data).
- There have been multiple hacks/losses in the Defi space due to oracle manipulation which suggests that we aren’t there yet.
Better Fiat on-ramps/off-ramps, are Centralized exchanges the single point of fiat on ramps to blockchains? [Source]

Updated last on the 29th of Jan 2021.

Inspired by Riva Tez’s following tweet.

Riva @rivatez

list of unsolved problems in biology taped to the wall of the synthetic neurobiology lab @medialab

Apr 26, 2019

A cautiously optimistic essay on OlympusDAO

Wed, 26 Jan 2022 05:03:26 +0000

This tweet is no longer available.

x.com/i/status/1485837498534154242

(3,3) is a meme that has taken crypto by storm, and while we might not be in the (3,3) phase of the project anymore, here is a look towards the future.

This article is a “the cup is half-full” look at Olympus, instead of the other side of the coin that seems to be popular. I only think of ohm as far as the decentralized reserve currency experiment is concerned any pivot from this idea moots the arguments that I’ve made in this essay.

There seems to be little love lost between most of CT and the downfall of Olympus. But if there’s one characterization that I don’t entirely agree with that has dominated the common parlance of Olympus talk is calling it a Ponzi, and I think it’s quite sad that the project itself tried to popularize that narrative on some level, you reap the seeds you sow.

What is a Ponzi, though? A quick search on google yields the following.

A Ponzi scheme is an investment fraud that pays existing investors with funds collected from new investors. Ponzi schemes are named after Charles Ponzi. … Ponzi used funds from new investors to pay fake “returns” to earlier investors. Ponzi scheme organizers often promise high returns with little or no risk.

By comparison, at least I’m unaware of Olympus ever promising high returns at little or no risk. For as long as I was active in the server till September (before its monumental run perhaps), the conversation in this case usually ran around what would happen in the case of a bank run and how most investors wouldn’t have lost a lot when considering the RFV of the token. Perhaps it was the APY marketing campaign that came closest to ohm ever being an absolute Ponzi.

But what went wrong? I think the Olympus October run and the November top almost echoed the levels of craze that were seen in the GameStop run which were seen in stock markets earlier this year. But I think the comparison mostly stops there. Speculative price runs are a breed of virus that eventually bring down any asset that they take hold of Doge, Shiba, Floki, time, hex and while you might think that I’m just adding “Ponzis” in this list, I’m sure you are aware of the bags that you have owned over the past year. It’s all reflexivity.

Ohm was actually countercyclical in the first half of its existence until September. You could chalk this up to a variety of reasons, but it could have been majorly attributed to the fact that ohm was actually a stables hedge, with APY embedded, since the treasury was all stables, and after they started diversifying the treasury into non-stables (eth) it took a more cyclical turn even though it was a fraction of the treasury. I think the thing that led most to ohms insane run and its subsequent downfall was its (9,9) feature. Anyone who has ever done any fundamental analysis in stocks or has looked at a financial balance sheet would tell you that Olympus was almost trading at 20-30 times its book value and *expanding supply* at this rate, which means quick dilution. I think you cannot allow lending-borrowing practices on the underlying expanding token (sOHM) at a 100% collateralization rate, something that would’ve saved ohm, perhaps also would’ve tapered its upward run, would’ve been a collateralization rate which was a function of its RFV. This would’ve removed the excessive amount of leverage that dictated and still dictates every downturn the token has seen. Another reason for the downfall was the immense concentration of the distribution in whales that were early to as much as one whale holding 85k ohm (about 120 million dollars at the top, which was sold for about 30 million dollars recently).

But…but… only early people really made money on ohm, it was only people who participated in the IDO? Even by the time I had left, only about 12 presale participants were still left. At this point I’m mostly convinced that the only way to really make money on assets, unless you are a trader, is to be early to it and while people in this space do make insane amounts of wealth through airdrops sometimes that is also perhaps precisely why we call crypto a generational opportunity or find some asset which has been in the scene for a long time and is undervalued and you wait for a people to see its value and even then in strictly monetary terms you are “early to it”.

But now that the token trades near its RFV and almost all leverage has been removed out of its system and the ohm holding concentration looks much more distributed than ever before, I personally am one of the biggest proponents and would like to see ohm succeed, perhaps not so as much for profit (I don’t own any bags) but because it would be the first successful “currency” experiment that was successful in defi.

Why do we need a decentralized reserve currency at all? Or what would it mean to achieve that target? From here on, I talk about the possibilities and you as the reader might have to allow me to wonder about a decentralized financial ecosystem.

A purely peer-to-peer version of electronic cash would allow online payments to be sent directly from one party to another without going through a financial institution - Satoshi Nakomoto, Bitcoin Whitepaper

That is the first line of the bitcoin whitepaper abstract. Bitcoin in its original form and its earlier narratives wanted to replace “money” and be used for our daily transactions and other uses that you commonly use money for. Most of its opponents always have drawn attention to bitcoins volatility to highlight its inability to be a currency in any form (something even ohm opponents draw attention to). But something that is an undeniable usage of bitcoin however small the volume is that bitcoin is definitely used for cross border payments because of its convenience, censorship resistance and usability. What would bitcoin have to accomplish to become an actual currency? Bitcoin would have to be held by multiple treasuries across nation-states across the world and allow it to become a legal tender perhaps if it really kicked off you would even at some point see bitcoin-backed currency (this is really outlandish and I don’t think so either). But the other reason bitcoin’s “currency” dreams never took flight was because of its network being bottlenecked on the scaling level and while you might talk about bitcoin L2s, none of them has seen the sheer level of adoption that merits it any mention here and bitcoin’s network security is under question given the trajectory that it is on so its dreams of becoming a currency are currently under question, The narrative that has caught on for bitcoin is the “store of value” narrative.

How is any of this relevant to ohm? It’s the idea of the need for a currency. The entire crypto ecosystem now hinges upon stablecoins and while stablecoins are good they are still a distorted representation of the currency-pegs we use, what is the current currency of the crypto ecosystem assuming we were living in a crypto world with no fiat anchor? Bitcoin. Because Bitcoin is used in a variety of places and every major exchange and CEX usually adds a bitcoin pair for the token immediately after the fiat pair. The gap that ohm is trying to fill is that of having a peg that is neither bitcoin nor a stablecoin, and in the ideal sense of the crypto ecosystem, so that it becomes the base token for a variety of assets to be measured in. While I’m aware that the risk-free value of $ohm is measured in x dollars, it’s exactly the idea of moving away ohm being backed by 1$ like every other algostable has tried to each ohm being backed by x RFV amount that the idea of being a reserve currency is about.

When did we all forsake ourselves? That we aren’t enthusiastic about the prospects of something like this being a possibility is something I often wonder about. Something that’s always been fascinating to me about decentralized finance was the sheer genius of the entire ecosystem. Within the years that it has grown and since the summer of defi when it went mainstream, we have now almost rediscovered the entire financial ecosystem almost replicating everything that is available and beyond while keeping our building blocks composable and in some cases even going beyond and discovering new primitives, which we might see adopted in mainstream finance. When you consider this, a decentralized reserve currency that replicates the gold/silver standard that we previously used to have is but one experiment that we should be excited about.

Have you ever wondered that you actually (3,3) with the currency that you use every day? now you would ask me what is the RFV of your token i.e your currency and while until the early 20th century we did have an RFV namely the gold standard today we don’t, our (3,3) lies entirely in trust and backing by the governments of the respective nations we live in, our currencies are backed by guns.

Ohm was in its expansion phase and I still think that it is in its expansion phase but something that is undeniable about the success of the project is the number of hands it has put itself in. Putting yourself in the hands of 80k holders (this number is probably only counting wallets, assume 40k unique holders for convenience) in <1 year of the launch of the project is quite the view to look at.

The (3,3) phase was more about the expansion of supply to me rather than participants engaging together, but that was still an important aspect of the system so as to not let the price of the token go below the RFV and this has been observed in a variety of ohm forks. But why was the expansion phase important? There are a few ways you bootstrap a treasury, you either do a sale of tokens or you do an expansion → slow growth of the treasury. And while the earlier might sound convincing, I recall a stablecoin project which had raised almost a billion dollars in the summer of 2021 and failed to maintain a dollar peg for the first few months and while it has since turned a fresh new page, it just proves that raising your treasury through a sale isn’t a panacea/universal solution. What excited me about the sheer growth of the ohm treasury even though you might call bonds analogous to TWAP sale of tokens, is the number of people that have bonded over time consistently with the system.

Something that has undeniably seen an uptick due to Olympus, is inter protocol diplomacy due to its Olympus-pro offering. And while a diverse treasury due to such token accumulation has its issues, something you can’t help but wonder in the case of taking that idea to its extreme is, let’s say crypto does grow 5x from here and a lot of protocols have been using OlympusPro to bootstrap their liquidity and OlympusDAO due to this holds their token in non-trivial amounts. There is a vested interest across all the protocols to help ohm maintain its peg lest the collapse affects their own treasuries or let ohm dilute these tokens for maintaining its RFV, and while this sounds like guerilla warfare, this vested interest in helping a currency maintain its status is how the US dollar reigns supreme as well. This is somewhat different than the curve-wars bribe meta that is popular for maintaining the pegs of a variety of stablecoins.

This article is not meant to be financial advice nor is it a call for you to load up on your bags because I don’t think the success of ohm would quite be measured in the multi-billion mcap that it has achieved or will have achieved. The final metrics for Olympusdao is its usage and the day even one user starts talking about a pair in “token-ohm” parity instead of “token-fiat/btc” parity I think ohm has achieved something worthy, and something that will be a case study in several universities across the world where economics is studied.

Thank you for reading this article, any feedback is appreciated.

Building Blocks of Primitive - Constant Function Market Makers [CFMMs]

Sat, 22 Jan 2022 07:47:21 +0000

Decentralized Exchanges (Dex), which provide a way for market participants to trade pairs of on-chain assets, have this additional feature of acting as an oracle for measuring the relative price for this pair of assets, and because of this can be thought of as Decentralized oracles.

Decentralized oracles are smart contracts that rely on users voting on particular prediction market outcomes. A final outcome is chosen via a social choice function, similar to how majority or weighted majority voting is used to decide outcomes in elections.

Decentralized oracles are becoming increasingly important and sought after sources of information because no one agent/entity/organization, i.e. a centralized source of information, can be trusted for its information completely. While this is not something wholly solved by the de-oracles, their design usually involves a “challenge” phase or/and provide users with rewards for reporting info correctly.

A fundamental problem when it comes to price markets is the “oracle problem”, the ability to have correct prices for an asset, or more generally, in the case of blockchains, the problem of providing external data (i.e. outside of the system) to a blockchain. One concern whenever using an AMM is that the price of an asset should accurately reflect that on a centralized exchange (assuming this to be the global price). AMMs are automated market makers and have no notion of this global price and aren’t adjusted accordingly; this price adjustment is usually left to arbitrage bots or traders who use these products. Why is the validity of this price being close to the global price important, though? Suppose an AMM generally has a negligible error or constantly reflects the same price as the global external prices. In that case, such an AMM’s price feed can be used as ground truth for various purposes and is said to be a “good price oracle”.

In this post, I will go over how CFMMs (a class of AMMs) can be used as price oracles and what interesting properties they have, while covering the following paper.

Angeris, Chitra et al. (2020): Improved Price Oracles: Constant Function Market Makers

The paper is a generalization of the authors work, An analysis of Uniswap markets. In this paper the ideas are generalized to CFMMs with the usage of convex analysis.

You can find a smaller summary of this paper, in this thread.

Tarun Chitra @tarunchitra

2020: Cambrian explosion in automated market makers — @UniswapExchange, @BalancerLabs, @ShellProtocol, @CurveFinance, ...

A question:

Is there a framework to understand why so many constant function market makers (CFMMs) work IRL?

Yes! Thread 👇🏾

arxiv.org/abs/2003.10001

Mar 24, 2020

Been on a mission to read more research in the AMM space. My favourite project, Primitive, is launching soon and is based on Replicating market makers [RMM]. There was a lot of precursing research to RMMs, and I’ve been lately making my way through the research. I can’t say that I have grasped every nitty-gritty of the paper or will be able to do justice to every aspect of the paper, but I’ll do my best to provide nuance or access to more resources. [Note: Convex optimization is hard]

Automated market makers were made famous by Robin Hanson’s “Logarithmic Market Scoring Rules for Modular Combinatorial Information Aggregation”, basically the LMSR market makers. These are functions that are extremely popular in prediction markets. (Refer to Gnosis’ blog or this one for understanding how LMSR’s actually work).

AMMs have gained immense popularity in the defi space as a primitive and have become essential building blocks. While the LMSR and its counterparts provided one kind of automated market makers, the cryptocurrency community has developed a different class called constant function market makers (CFMMs). Uniswap’s V2s constant product AMM quickly comes to mind.

Since the CFMM paper was written in march 2020, DEX volume has increased from roughly 10 million per day to a few 100 million, reaching even a 2 billion dollars of trade volume every day [As of 15th Jan 2022]. This has led to Uniswap (the most popular dex) being used as a price source.

A natural question to ask here is, Is Uniswap a good price oracle for every type of asset pair possible? i.e. is Uniswap’s AMM design optimal for all assets? The answer for this is no. Stablecoins arent usually best handled by Uniswap and have been addressed by better mechanisms through curve. Although recently since the introduction of Uniswap v3, it has come to light that may be in the current state of the AMM landscape no one AMM is superior to the other when it comes to stables and that we might be hitting the upper limits of asset-based AMM design, as seen follows. In the absence of superiority of product the usage falls back to fees, and Uniswap v3 has been seeing steady growth of its stable coin market share since it introduced the 1bp fee tier.

Ryan Watkins @RyanWatkins_

Pretty incredible that Uniswap's stablecoin market share vs Curve has rocketed from 43% to 89% in just two months since it introduced its 1bp fee tier - the most popular stablecoin pools on Uniswap now (h/t @tomhschmidt).

The pressure is now on Curve to cut fees (4bps).

Tom Schmidt ＞|＜ @tomhschmidt

Sorry, but Curve wars are dumb.

1) These features should've been built into Curve natively from the start. A product miss has spawned this entire industry.

2) This glosses over Curve losing stablecoin swap mkt share. You're bikeshedding while the shed is burning down behind you

Jan 19, 2022

The paper’s contributions are as follows:

A complete and general framework for analyzing CFMMs
Provided sufficient conditions
- Agents are incentivized to have the CFMM report the correct prices of an asset (the implication of this being that AMMs can act as sound decentralized oracles given the conditions are satisfied)
- A malicious agent has no way of draining the assets of a CFMM by only trading with the given CFMM.
A simple derivation for the total asset value in a CFMM as a function of external market prices

While a mathematical treatment of the entire paper here would do justice to it, I will not go into the depths but instead talk about the definitions in short and what they help accomplish/prove in the paper, so we can also take away the main results. I think the math in the paper is beautifully written and straightforward.

The definitions of the paper help you make a mental framework around market makers and a neat problem formulation to work with if you are trying for optimal trades or are interested in the CFMM design space.

Constant Function Market Makers

Trading Function

Now, these definitions break down an AMMs functionality into two parts. The CFMM definition itself defines the domain to the range $R$ and secondly, the trading function, which is a representation of a simple trade. Today we only deal with both the input and output trades as vectors of size 2, wherein you have the exchange of (token 1, token 2) → (token 1, token 2). We also deal with larger vectors when it comes to exchanges that have routers wherein you don’t have a direct pool from token 1 → token 2 and choose to hop between $n$ pools to get the trade done, in this case, the size of the vector of both the input trade and output trade will be larger. In the future, we might even have multi-coin tuples where you trade a set of tokens at once with some other set of tokens which saves you the number of distinct transactions that you might have to do.

The trading set is further defined to have the ability to compare two trading functions, and it offers a way to work with the trading functions elegantly. An interesting consequence of this definition is that constructing equivalent CFMMs with trading functions with properties of interest becomes easier.

Convexity, Path Deficiency and Path Independence

It is assumed that trading sets are closed convex sets. What does this allow? The field of convex optimization has been thoroughly developed over the past century. A convex definition allows agents to find an appropriate solution that maximizes their payoff to an optimization problem over the trading set. There is no CFMM used in practice whose trading set is non-convex (which is important for generalizing this result).

The Reachable reserve set is the set of reserves that can be “reached” from the current CFMM reserves by performing a trade. “Reached” in this context means the value of the underlying reserves.

Practical CFMMs don’t satisfy path independence, but what does path independence mean? While I earlier thought path independence (from what I understood) was if you arrived on the same values of for the reserves of token 1 and token 2 no matter what path i.e route of assets you take. If you think of the variety of assets that are present in a CFMM, then going from any one asset to some asset in the system will have a variety of paths i.e token 1 → token 2 directly or breaking this down into doing multiple small transactions from token 1 → token 2. Now this is something that is not practically possible because of fees, multiple transactions accumulate the fees so you don’t end up with same amount of token 2 as you would if you would directly do it.

Path Independence from the paper, “An analysis of Uniswap Markets”

A slightly more general property to talk about when thinking about path independence though is path deficiency, path deficiency instead of talking about the path independence instead makes guarantees about the underlying reserves and this is a property satisfied by all known CFMMs. No trade in a CFMM can make the reserves larger than the current one, i.e they cant make assets out of nowhere.

Optimal Arbitrage and Marginal Price

The optimal arbitrage trade is given as follows,

Here $\Lambda$, is the output tuple, specifying how much of each coin in the ith entry of the tuple after the trade, similarly $\Delta$ is the input tuple, specifying how much of each in the ith entry of the set before the trade. An optimal arbitrage would try to maximize the difference, as seen in (8). Thinking of an example here you could consider (token1, token2) as your tuple and in an arbitrage, you would want to start with $(x_1, x_2)$ and end up with $(y_1, y_2)$ where the sum of these differences i.e $(\text{asset price of token 1}) \cdot y_1 - x_1 + (\text{asset price of token 2}) \cdot y_2 - y_1$ is highest.

While generally, it could so be possible that solving (8)/(9) does not give the best possible strategy due to market inefficiencies or otherwise or an arbitrageur could break this trade into smaller trades and these could lead to higher benefit. Something that the paper proves is that, for path deficient CFMMs there is no strategy for which an arbitrageur can get a higher payoff than solving (9) and that solving (9) gives the highest payoff possible.

There are some optimality conditions that need to hold for (9) to be true, which lead to a simple definition for reporting the price (would suggest people interested in this condition to check the paper pg 15). The definition of this asset implies that whenever the price of the CFMM is mismatched to that of a reference market, an agent is always incentivized to make “free money” by only trading between these two markets which would also bring the two prices to parity.

While the rest of the paper is beyond the scope of this discussion, there are some results worth pointing out:

The next section elicits the problem to be solved for find the total value of the reserves in a CFMM, ultimately posing a single variable convex optimization problem as follows

Which when solved for the case of uniswap with zeros fees and a special case of two assets leads to the solution

I wonder how this applies to the Uniswap v3 model which isn’t exactly a CFMM due to its formulation of concentrated liquidity (the ticks fractionalize a continuous curve into discrete parts), but each tick can be seen as a CFMM. I’m excited to write about Replicating Market Makers next, these are CFMMs whose portfolio value function matches the payoff of a desired product, in simple words owning liquity in these would be similar to the payoff you would get if you bought the desired product. Primitive’s RMMs are designed to match the payoff of a covered call option. I’ll also be looking into some papers which formalized MEV recently and some others which talk about the importance of liveness in blockchains.

Got a long way to go, Thank you for following along.

0xEmperor

Wildcat: Banking but Better

The Allure of Credit: From tradfi to crypto

Wildcat: The Protocol

Against Wildcat: Challenges for Onchain Undercollateralized Credit

Conclusion

Understanding "An Analysis of Intent-Based Markets"

Intents in Practice

An Auction Theoretic Model

No Entry Costs

Costly Entry

Exponential Distribution

Uniform Distribution

Pareto Distribution

Costly Effort

Effort-Dependent Prices and Costs

Cost of Effort and Congestion

Equilibrium Effort

Specific Example: Uniform Distribution

Welfare as a Function of Effort

Optimization Model

Framework and Assumptions

Social Welfare Maximization

Dual Approach and Dutch Auction Mechanism

Extension: Congestion

Off-Chain Compute Is All You Need

Introduction

How we got here:

Towards Unification of Off-Chain Compute

Same Same But Different

A Proof-Based Future

—————————Dont Trust, Verify—————————-

Blockchains: Unlocking a Privacy Layer for AI

An abundance of AI applications

Chatbots

Programming

Law & Medicine

Maths

The privacy question for AI

Multi-Party Computation on Blockchains

MPC with Transformers

The Transformer architecture

Blockchains for Inference only?

Conclusion

A Brief Intro to Coprocessors

Zk-coprocessors

Axiom

Risc Zero Bonsai

Alternative Solutions

MPC (Multi-Party Computation)

Secure Enclaves / TEEs

A brief comparison

Applications

Dynamic Control for Lending Protocols

ML applications

Perpetual swaps and Options

Conclusion

A List of Open Problems in Crypto - II

Table of Contents

Decentralized Finance [Defi]

Open Research Problems - Mechanism Design

Open Problems in MEV

Some Problems to Upgrade Zero Knowledge

Other Problems in Crypto

An Incomplete Primer on Intents

Understanding an Intent

A few examples from the past and present

Cowswap

Uniswap-X

Towards a formal definition of intents

Intents as a boon for UX

A conversational Intent flow

Another DSL Architecture - Essential

Anoma

Conclusion

References

Designing Multi-Dimensional Blockchain Fee Markets

Exploration of Fee Markets

Rollups and Data markets

Parallelization