Bringing HFT To DeFi With Doug Colkitt
Welcome to another issue of Front Month, a newsletter covering the biggest stories in exchanges & market structure every Friday. If you have questions or feedback, please reply to this email or find me on Twitter. If you like this newsletter and want to follow the exchange industry with me, please hit the Subscribe button below & be sure to share with friends & colleagues:
This week I had the chance to interview Doug Colkitt, a former high frequency trader at Citadel & founder of decentralized exchange CrocSwap. I’ve been following Doug’s excellent work ever since he started sharing insights on Twitter, and really enjoyed his recent appearance on Patrick O'Shaughnessy’s podcast, Invest Like The Best. Doug & I had a great conversation about the nuts & bolts of professional trading, his transition from traditional high frequency trading to crypto, and how his new DEX plans to move the industry forward.
Please go give Doug a follow on Twitter & thank him for sharing his expertise!
You got your start on Wall Street when you joined Citadel out of college. What attracted you to the industry & that firm in particular?
I joined Citadel in mid-2008 after interning with a few different banks in quant-type roles during college. At the banks, my roles were very structured – I was put in charge of one small, niche thing within the larger organization. I found prop shops & hedge funds like Citadel more attractive because they were very entrepreneurial – one person could take on more responsibility & not be as burdened with bureaucracy. The choice came down to two firms – Citadel & Global Alpha, a quant-type hedge fund within Goldman Sachs. Global Alpha was a well-known fund that had focused on stat-arb type strategies since the 1990s with good success.
Looking back, I think I made the right choice with Citadel, especially when you consider the rise of high frequency trading. High frequency strategies were really starting to ramp up around the ‘07/’08 time period and involved pushing the limits of trading software & technology. My background was in computer science, so I felt I could add more value to a growing field like HFT with such a tech-heavy focus. It turns out that around this time, most quant funds started melting down with the rest of the market while high frequency traders & market makers were printing money. It was an interesting experience working for a very successful, stable trading team during 2008 & ‘09 amid a very unstable time for nearly every other part of the industry.
What did your role in the HFT group look like?
When I was there the HFT group was a very academic environment. I joined the team as a researcher, which meant building different strategies that our traders would then go and execute. Researchers were treated a lot like traders in the sense that we tracked daily P&Ls and had to attribute parts of our P&L to certain strategies. There was also a team in charge of generating the signals that would be used as inputs into our models.
For most of my time at Citadel I was running strategies on Asia Pacific equities, which I had volunteered for because I didn’t have kids at the time & could work around Asian market hours. Our job was to predict prices over short time periods – anywhere from a few milliseconds at a time to as long as ten minutes.
The nice part about my role was the robust datasets that were available to our team. If you’re a macro trader, for example, it’s very hard to predict prices over multi-year timeframes from a statistical standpoint. If you’re trying to predict prices over a one second timeframe, however, you have huge pool of short-term data points to incorporate into your strategy, and you’re constantly getting new data to validate your models as time goes on. Although market news doesn’t come out every second, the order books of the stocks you’re trading & most other assets are consistently changing every second & become your universe of data to work with.
How did your career interests change as you got up to speed & progressed within Citadel?
I was at Citadel for 2-3 years and found it a great place to learn, but I wanted to go out on my own and build something from scratch. When I left Citadel I set out to build my own high frequency trading system from the ground up, and quickly learned how much I took for granted as a researcher for the firm. At Citadel you were able to see a few specific parts of the operation but never the full picture, which prevented employees from walking out the door able to recreate everything Citadel was doing. So I had a lot of learning to do to get my own system off the ground.
To start I had to pick a market to trade, ultimately landing on CME futures as my asset of choice. I liked futures over other asset classes like equities because they were much less fragmented – futures traded on one exchange with very little if any off-exchange activity, allowing me to see the entire market in one place. I figured I could connect to CME and profitably trade index futures, which were some of the biggest, most liquid instruments in the space. I collected a bunch of historical order book data & built simulators to test out different strategies, then created my own signals to incorporate into those strategies.
I spent some time back-testing & refining what I had built, then moved on to the hard part – connecting to the exchange itself. I had to build all the infrastructure to receive and parse CME’s data feed, talk to their exchange gateway & interface directly with their matching engine. When most traders interact with the exchange they’re connecting through an intermediary & not directly to the exchange itself. Because I was going down the DMA (Direct Market Access) route, I had to get a plethora of exchange approvals to start trading & have multiple risk checks performed on my system to make sure I wouldn’t errantly crash the exchange. I spent just as much – if not more – time connecting to the exchange as I did building my core trading strategy. After all the checks have been cleared I could finally co-locate in the exchange’s data center & start trading.
How did your strategy pay off initially? Did you see the kind of success you were hoping for trading on the CME this way?
I started trading in mid-2011 when there was still a fair bit of residual volatility left from the financial crisis, which when combined with the European sovereign debt crisis made the environment very favorable for the HFT space. I was never expecting to be as good as the Citadels & Jumps of the world, but volatility turned out to be high enough for the apex predators to eat & still have enough opportunity left over for smaller players like me to do well.
HFT models are roughly similar across groups with each shop building their own secret variation of that model to succeed. To the extent that your model is sufficiently unique or different from everyone else, you’re more likely to capture differentiated profit. In high volatility environments, the magnitude of that differentiated profit is much larger than in calm markets, because more people are in the market & more volume creates more opportunities. Volatility inevitably died down over time & although I was improving my models as my strategy matured, there just wasn’t enough out there to capture & I started to focus on more medium-term trades over the short-term, high frequency type approach.
How did the makeup of the exchange itself impact your trading strategy?
We’re getting into the more technical depths of market structure here, but there was a, call it a quirk about CME’s infrastructure that made it harder for smaller players to compete. CME had a bunch of gateways where trading firms could link into the exchange, and those gateways weren’t strictly first-in, first-out back when I was trading. CME operated a round-robin style ordering methodology that would determine which gateways would be layered into the matching engine before others, and if your gateway was chosen first, you’d be first in line to capture an opportunity.
To take advantage of this, larger trading firms would buy as many gateways as they could & link their system across every gateway to increase their chances of being picked by CME to be first in the queue. I couldn’t support that kind of ramp-up in infrastructure spend at the time, so I was less likely to get winning trades than the bigger players regardless of the strength of my strategy.
CME ended up changing their ordering methodology shortly after I stopped trading there to put timestamps on every message coming into their exchange so it could be fairer, but that was one example of the kind of quirks traders had to work around to be successful.
With calmer markets & less opportunity at CME, where did you look next to keep your business afloat?
After I wound down the CME trading system I started looking for ways to put my strategies to work in smaller, less efficient markets. I randomly connected with some local Turkish partners on the ground who didn’t have the quant background but had connections with onshore brokers & knew that Borsa Istanbul was upgrading its electronic trading systems using Nasdaq technology. I found Turkish equities attractive as an asset class because of my connections there & the fact that, like CME futures, everything traded on one venue.
I decided to co-locate & start trading in Turkey in mid-2016, leveraging most of my existing CME infrastructure to get up to speed. My plan was to get a handle on trading profitably in Turkey, then slowly branch out to other emerging markets where the trading environment was similarly attractive. As far as volumes & volatility went, Turkey was better than expected & about on par with Australia in size & activity. I was able to trade there with better success than what CME was offering at the time.
This is about the point in your career when you started focusing more on crypto as an asset class in addition to Turkey, right? How did your crypto interest grow & what did you do to pursue opportunity there?
I first heard about Bitcoin in late 2009/2010 and had been interested in the concept from an intellectual perspective for some time. Employing the same strategy of taking my existing trading infrastructure to less efficient markets, crypto was an asset class I did start to explore. The challenge with crypto was the fact that the infrastructure surrounding those markets wasn’t as mature as CME or Nasdaq or even Borsa Istanbul. Crypto markets don’t have colocation, they generally trade on the cloud, and their trading protocols aren’t as optimized. My existing trading stack wasn’t perfectly set up to interact with crypto & my strategies – some of which relied heavily on speed – wouldn’t be as effective.
Then, randomly in 2020, I read Paradigm’s “Ethereum is a Dark Forest” article that talked about MEV (Miner Extractable Value) and found it extremely interesting. Just to see if I could do it, I built a prototype MEV bot while continuing to trade Turkish equities full-time. I found that it was much easier to get started in the crypto space via this MEV bot than it was to enter the HFT space – my returns-per-effort remained quite high as my crypto exploration progressed. After a certain point, it became more profitable to pursue crypto trading full time than it was to maintain my relatively stagnant Turkish trading business – markets were slowly starting to grow more efficient there as well.
It’s worth keeping in mind that the DeFi space as a whole was growing by thousands of percent per year at the time, so even if you were just treading water in terms of competitiveness you were still doing pretty well with a DeFi-focused business.
How should readers think about MEV as it relates to the broader crypto ecosystem? Would you consider it comparable to how latency arbitrage works in traditional HFT and market structure?
There are a ton of different types of MEV strategies, but the general idea is if you can control the sequence of blocks executing on a particular blockchain you can extract profit in a competitive, zero sum game kind of way. That’s similar to latency arbitrage & high frequency trading in general because there’s a focus on queue position & a competitive race to be first to a trade.
What’s different between the two is that with HFT you’re almost always reacting to a trade that’s already happened, whereas with MEV you’re frontrunning a transaction before it even executes. In TradFi everything is atomically ordered – by the time I see someone else’s transaction, it’s already been processed through the matching engine. In a blockchain context, that’s not true because blocks only happen in Ethereum’s case every 12 seconds. If someone trades on Uniswap, for example, that trade is broadcast on a peer-to-peer network and I can see that it’s in the queue. I can then play games with how blocks are constructed to frontrun that transaction. This concept of seeing & being able to out-run a trade before it reaches an exchange doesn’t exist in TradFi.
Would you say that MEV adds value to the crypto ecosystem in any way? If so how?
Good question – it’s hard to see how frontrunning adds any value to the space. Some people will disagree with me, but I think all high frequency trading is value-add. HFT market making directly adds liquidity & even price taking is value-add because you’re helping make markets more efficient by acting on signals & moving markets to a more correct price. Frontrunning, on the other hand, is pure exploit & I don’t think that is value-add to any industry. This isn’t a “holier than thou” kind of view, it’s just sharing the reality that MEV is a pure PvP exercise – someone sized their trade wrong & you can come in and take advantage of that.
I’ll give you an example – let’s say somebody goes and makes a large buy on Uniswap which moves the price of the pool they’re trading with. Uniswap has slippage parameters outlining how much traders are willing to move prices before canceling a transaction. Let’s say this trader sets their slippage too high. As an MEV bot I can buy the token ahead of them, run up prices to the maximum of what they’re willing to pay, and then run prices back down all in the same block. The fact that I had engaged my MEV bot changes nothing about the market – prices are the same before & after the trade. I just extracted money out of this one person who tolerated too much slippage.
The best thing I can say about MEV is that it’s brought a lot of smart people into the crypto space.
At what point in your crypto journey did you decide to transition from a trader to an exchange operator? What went into that decision?
I started exploring the idea of building a DEX when Uniswap v3 came out in early to mid 2021, which offered a very different mechanism of decentralized liquidity provision. In a classic AMM model like Uniswap v2, liquidity providers can’t express a price view – they provide liquidity at prices ranging from zero to infinity & the only way prices move is if someone trades against them. This is very different from the TradFi model where liquidity providers are very active contributors to the price discovery process. Market makers are constantly adjusting their quotes based on different signals & inputs to their models that predict prices over short time frames – you always hear the statistic that these traders cancel 99% of their orders because they’re always moving their quotes around. Because traditional AMMs didn’t let liquidity providers express a price view, I always had a skeptical view towards them.
When Uniswap v3 came out, however, trading started to look a lot closer to a limit order book environment, which piqued my interest. As I started thinking of different ways I could trade & build new strategies in this environment, I kept finding features I wished existed within Uniswap’s protocol – things that would make my life easier as a trader or make trading & price discovery more efficient. After a while, instead of complaining about the features that didn’t exist, I decided to build an exchange that offered the features I wanted instead.
Staying on the AMM model for a second, would you say it’s accurate to label most AMMs as primarily off-ramps for toxic flow?
Yes, I would argue that’s very close to 100% accurate. The problem with AMMs is that liquidity providers aren’t sophisticated. I’ll use traditional markets as an example – if I decided to go make a market in Microsoft stock on Nasdaq, and I always posted a bid & an ask and never changed my position, I would lose money very quickly. People would take my liquidity & run me over when it’s in demand & wouldn’t pay me for it when demand wanes. The same logic applies to AMMs in DeFi.
Naturally, when you look at the makeup of order flow on decentralized exchanges, ~50% of volume is made up of bots who watch centralized exchanges & arbitrage against the DEXs when prices move out of line.
Another problem with AMMs is their fee structure. AMMs technically don’t have bid-ask spreads – they have one price & use their trading fee as the determinant of the bid-ask spread. Uniswap v2’s trading fee for a long time has been 30 bps. It doesn’t really make sense for a market’s bid-ask spread to always be 30bps – spreads should widen in times of high volatility & toxicity and narrow when it’s low. When you combine all these issues together it creates a situation where most AMM liquidity pools aren’t worth the risk to support.
So with Uniswap v3 a step forward but still not everything you wanted as a trader, you decide to build your own DEX, called CrocSwap. Help us better understand how CrocSwap improves on the existing DEX offering & who your target audience or customer is – institutional or retail?
I think the core way that we improve on the existing model is reducing friction for active liquidity provision. Uniswap v3 allows traders to provide active liquidity, but the gas cost associated with adjusting quotes & fee tiers is still too high. As we’re building CrocSwap we’ve focused on optimizing those types of operations to make it much more attractive for traditional high frequency traders to jump in. I think this will make markets much healthier long-term because we’ll start to see less stale liquidity provision in the DeFi space.
Another area we’re looking to improve is the dynamic fee tier model. As I mentioned earlier, Uniswap v2 offers fixed fee tiers which don’t account for changing market conditions. UNI v3 improves this by offering pools with different fee tiers but it’s still too costly to constantly move between them. At CrocSwap, in addition to reducing the gas cost of changing tiers we’re looking at using oracles to dynamically adjust fee tiers based on outside market data.
To answer your question re who our target customer is, I think it’s a combination of institutional & retail. We’ll still have UNI v2 style AMM pools available to support smaller tokens that subsidize their liquidity, which I think is valuable for those kinds of projects. I do think however that concentrated liquidity will bring many more institutional players into the DeFi LP game & CrocSwap makes it easier for them to do so.
How are you planning to attract volume & market share to your exchange? Who would you say are your main competitors?
The DEX space is still growing at a solid pace but competition has matured quite a bit in the last few years. On Ethereum you have the likes of Uniswap, Sushiswap, Bancor & Balancer all trading the same tokens which naturally drives them to compete with one another. Some exchanges put too much focus on LPs, others on swappers, but the reality is it’s a two-sided market & both sides are important.
CrocSwap is focused on LPs by protecting pools from MEV bots to help them trust the yields they’re seeing, and is also focused on swappers by providing competitive execution quality. In DeFi there’s a lot of capital that runs through aggregators who look for best execution across exchanges, so as long as you grow to a size where aggregators start monitoring your quotes, volume should come.
How would a native CrocSwap governance token play into the competitive dynamic? Do you have plans to launch a Croc token?
We don’t have any specific tokenization plans at the moment, but the idea behind a DEX token launch would be that it’s a great way to quickly bootstrap a network & attract user interest. I think about a governance token like equity in the protocol itself, just like Nasdaq has an exchange & shares of its own stock that trade on its exchange. Over time the protocol generates revenue that gets proportionally paid to token holders, so the value of a DEX’s token should over the long term reflect the cash flows holders would expect to receive.
DEXs also exhibit network effects – the more people interact with it, the more valuable it becomes to each subsequent user. On the flip side, however, startup exchanges struggle with the chicken & egg problem – even if it has a strong product to offer, the fact that it’s small makes it less valuable to prospective users. DeFi as a whole has been very good about compensating early users for overcoming these kind of barriers, in the form of tokens or some other kind of reward. A native DEX token would help accomplish this.
What do you see as a current or future real-world use case for a DEX like CrocSwap & DeFi as a whole?
As DeFi & the blockchains that support them improve their performance and user experience I very much think we’ll start to see DEXs compete with centralized crypto & even TradFi exchanges in the future. We’ve seen things like this happen in the past where technology that was first viewed as nothing more than a toy quickly becomes much more mature. If you have the mechanisms in place to attract traders from all over the world, your market quality is going to improve to a point where you can start to compete with any other exchange out there.
In terms of real-world use cases, as the technology improves I don’t see why we couldn’t migrate traditional assets to this decentralized model. There’s no reason you can’t tokenize a stock or a commodity future & put it on a DEX at some point down the road. There are obviously regulatory issues around this which will take longer to sort out, but you only need one jurisdiction somewhere in the world to pave the way for others to follow.
I’ll share one analogy I think applies – encryption itself was very heavily regulated until the late 1990s. US companies weren’t allowed to sell software overseas that had encryption in it because it was labeled as a sensitive technology, which came with strict export controls. That might’ve made sense in the mid 1900s but as technology evolved the law became outdated & it changed. We could see something similar happen to the protocols that are growing in DeFi today.
Thank you for reading this issue of Front Month. Word of mouth is the #1 way others find this newsletter - If you liked this week’s content, please consider sharing with friends & colleagues. Questions & feedback can be sent via email or Twitter.
Disclaimer: I am not a financial advisor. Nothing on this site or in the Front Month newsletter should be considered investment advice. Any discussion about future results or projections may not pan out as expected. Do your own research & speak to a licensed professional before making any investment decisions. As of the publishing of this newsletter, I am long ICE, CME, TW, SPGI, NDAQ and VIRT. I am also long Solana.