I spent 2 years rebuilding my algorithmic trading platform in Rust. I have no regrets. | by Austin Starks | Dec, 2024

0
I spent 2 years rebuilding my algorithmic trading platform in Rust. I have no regrets. | by Austin Starks | Dec, 2024
Austin Starks
The Rust mascot

One day, after a particularly arduous coding session with Rust, I decided to vent my frustrations with this language on pen and paper (or keyboard and pixel?). My rant ended up going viral.

The programming community was as divided as Korea in 1945. While some people called me a complete idiot with no skills, other people came out of the woodworks and vehemently agreed.

A comment about my “low IQ”

However, after sticking with Rust, iterating on my trading platform, and learning some new things, I’ve had a change of heart. The Rust programming language really isn’t that bad.

I honestly kinda like it. Here’s why I changed my tune.

Now before I start, I want to clear up some misconceptions from my last article:

I did NOT pick Rust because it was “cool”.

I am building an algorithmic trading platform. My first version of the platform, NextTrade, was built using TypeScript.

After building the TypeScript version for over two years, I had to give up. This version suffered from major issues, including being too slow and inflexible.

For example, running genetic optimizations on my Macbook Pro could easily crash my entire computer. And, if the population size was small, it would take days for 100 generations to pass for a complex strategy.

The platform was just not scalable.

I decided to open-source it and rebuild the entire core trading logic in another language. My top contenders for a language included:

  • Golang: Something that I was familiar with from my day job. Golang is lightning fast, easy to use, and scalable. Development speed would’ve been amazing.
  • C++: The traditional pick for algorithmic trading platforms. I was curious about trying it (especially because I had dreams of working at companies like Jane Street). But the things I read about it online turned me off.
  • Rust: “The most admired language among developers” according to GitHub. The article points out correctly that Rust is lightning fast, highly concurrent, and great for performance-critical backend systems (like an algorithmic trading platform).

After evaluating my requirements and doing some research online, I ultimately decided to use Rust to rebuild my platform.

And in the end, I’m now happy with my choice. I built a platform that’s scalable, configurable, and lightning-fast. It enables anybody, even beginners and non-technical investors, to deploy their own algorithmic trading strategies.

But in the beginning, I hated the language. And here’s why.

For the entire first year that I worked with Rust, it felt like I was learning something new.

As someone who came from Python to Java, to JavaScript and Go, this was a completely new experience to me. For every language I’ve learned until Rust, I felt like I could learn on the fly.

I quite literally shipped Go code the first day I saw it.

With Rust, this couldn’t be further from reality. While I knew the language was different, and I came into it knowing it had novel concepts, I didn’t understand just how different it would be.

And I had to fight the language tooth and nail for a long time. Certain things that was natural in languages like Python and TypeScript simply were nightmarish (if not impossible) in Rust.

For example, in my last article, I talked about the “Horrendous, verbose, unintuitive syntax and semantics”. I gave an example of a helper function, whose purpose was to commit a transaction with retry logic.

Rust “run_transaction” helper function

In TypeScript and Go, writing such a function would take a minute or two. In contrast, I was writing the Rust function for well over an hour, and then I abandoned the idea of a helper function entirely.

I’m quite simply not used to fighting a programming language.

Even Java, which won’t compile if you misplace a single semi-colon isn’t as strict as Rust. Semantics that work well in every single other language that I’ve ever seen quite simply don’t work in Rust.

And I hated that.

This shift extended beyond simple syntax and semantics. For example, while you get stack traces for free in languages like Java, you have to explicitly “activate” stack traces in Rust using environment variables.

# How to enable stack traces in Rust
RUST_BACKTRACE=1

With all of these quirks, if my goal was simply to be productive, a language like Rust is a frustrating experience.

But finally, after I continued iterating on the app, learned the quirks, and shipped feature after feature, I’ve come to a newfound appreciation for the language.

Here’s why.

With all of this being said, while working with Rust, I’ve learned that there’s a huge difference between building an app and maintaining one.

And maintaining a Rust app is like getting kisses by a butterfly.

It’s just so, pleasant. Aside from the occasional serde serialization/deserialization issue (where I might’ve misnamed an attribute or forgot to put Option<Type>), maintaining a Rust app is so easy.

Extending functionality and maintaining a full scale app is a breeze. I LOVE rust enums, and pattern-matching is a language pattern descended from the heavens. And the fact that I’ve never run into an NullPointerException in this language is not a coincidence.

Even Golang can’t say the same thing.

And, as I mentioned in my last article, Rust is fast. Like really fast. Genetic optimizations, which took hours or days in TypeScript, took seconds or minutes in Rust.

The difference was night and day.

Finally, if you’re like me and struggling to get the hang of Rust, here are some tips and tricks for becoming a productive Rust programmer.

1. Use enums correctly

When I first started using enums in Rust, I used it like I did in TypeScript.

// An example of poor utilization of Rust enums. The Conditions themselves
// could've been an enum
#[derive(Debug, Serialize, Deserialize, Clone, PartialEq)]
pub enum ConditionType
Base,
And,
Or,
Multi,

#[derive(Debug, Serialize, Deserialize, Clone)]
#[serde(rename_all = "camelCase")]
pub struct AbstractCondition
pub _id: Option<mongodb::bson::oid::ObjectId>,
pub name: Option<String>,
#[serde(rename(serialize = "type", deserialize = "type"))]
pub condition_type: ConditionType,
pub lhs: Option<Indicator>,
pub rhs: Option<Indicator>,
pub comparison: Option<Comparison>,
pub conditions: Option<Vec<AbstractCondition>>,
pub form: Option<FormControl>,
pub value: Option<f64>,

This was incorrect.

Unlike other languages, within Rust, enums can contain attributes. This makes them incredibly powerful for modeling complex data structures and enabling cleaner, more expressive code. For example:

// An example of enums done right in Rust
pub struct BacktestEventEmitter<'a>
pub market_queue: VecDeque<MarketData>,
pub event_queue: VecDeque<Event>,
pub accepted_orders: Vec<Order>,
past_orders: VecDeque<Order>,
pub accepted_transactions: Vec<Transaction>,
past_transactions: VecDeque<Transaction>,
pub brokerage: &'a BacktestBrokerage,
event_history: Vec<Event>,

pub struct LiveEventEmitter<'a>
pub event_queue: VecDeque<Event>,
pub brokerage: &'a Brokerage,
use_real_time_market_data: bool,
user_id: mongodb::bson::oid::ObjectId,
pub is_paper_portfolio: bool,

pub enum EventEmitter<'a>
Live(LiveEventEmitter<'a>),
Backtest(BacktestEventEmitter<'a>),

This structure allows you to handle distinct cases (like live and backtest event emitters) while keeping their associated data and functionality encapsulated within the enum.

2. Use abstract helper functions sparingly

Unless you’re repeating the same block of code 10 times throughout your application, don’t use helper functions such as my run_transaction.

Doing so will frustrate you to no end. The language isn’t like TypeScript, where everything can be a helper function. While it is possible, save your past-self some time, and let your future self worry about refactoring the code.

Chances are, you won’t ever need to.

3. Seek help from outside of Reddit and StackOverflow

If you’re struggling with something in Rust, don’t go to Reddit.

There are (allegedly) Rust forums where the people are genuinely helpful and want you to be successful. They’re not going to answer questions like “how do I get MongoDB to work?” with answers such as “use Postgres instead”. That’s nice.

Me personally, I still rely heavily on using Large Language Models for help. Now that I understand the Rust language a little more, I can articulate my requirements better, and in turn, get better answers from these models.

If you’re like me, and love to use Claude and ChatGPT to help you write code, just make sure you understand why it is making the decisions it is. If you see a compilation error, don’t blindly copy/paste it. Read the message and try to identify what is going wrong.

And if you still don’t understand it, ask AI to explain it. Eventually, you’ll learn.

4. Use clone(). Always.

Okay, maybe not always, but you’re almost certainly not going to notice a difference in performance. Your code will just work more often.

Rust is a polarizing programming language with many distinct advantages and disadvantages. It is unlike any programming languages I’ve ever worked with, and I’m relatively proficient in half a dozen. This makes the learning curve steep for anybody not willing to sit down, read the Rust book, and watch half a dozen tutorials on YouTube.

For people like me, the process of learning Rust clashes with my personality. I like to move fast and break things, and using Rust is like trying to run a 100 meter dash in a 4ft deep Olympic swimming pool.

But these constraints come with good reason. And when you get used to it, some of its quirks like pattern matching, options, and result types are kinda nice.

And, its lightning fast. I know for a fact that if I had chosen to write the language in Go, I would be filled with dread. “What if it could be even faster?” I would think.

And if I had done C++, I may have had to scrap it altogether after my program crashes for the 18th time due to a segfault 😂.

After learning it, I feel like I can build almost anything. I’ve successful used it to build a lightning fast, highly configurable algorithmic trading platform. The struggle of learning Rust was worth the effort in the end.

So I take back what I said in my last article. I have no regrets.

link

Leave a Reply

Your email address will not be published. Required fields are marked *