Ready to write Python code 4x faster?
I've spent the past 3 years of my life building a spreadsheet. Day in, day out, rows and columns. Pivot tables and graphs. Formulas and importing and exporting.
Building a spreadsheet is how I plan to spend the next decade of my life – which is a somewhat sobering thought. A decade ago, I was still in high-school, and was working up to wearing jeans to school for the first time.
The next decade feels like a long time. I'm wondering why the hell building a spreadsheet takes so long.
I could build that in a weekend
There's an infamous HN comment on the launch of Dropbox where someone argues that "you can already build such a system yourself quite trivially by [list of steps well beyond the reach of most users]." Somewhere deep within the psyche of every engineer is a belief that they can solve any problem much quicker than it's currently being solved. "I could solve that in a weekend!"
And I'm no different. I'm not being fair when I compare the complexity of my product to the complexity of other friends products. I know I'm judging a spreadsheet, whose complexities I'm very familiar, to products where I have no insight.
So let me go on record: every problem is secretly hard, and every product has hidden complexities not visible from the outside.
Here's the thing, though. Spreadsheets are not secretly hard. They are obviously hard. The complexities aren't hidden from the outside. Spreadsheets look complex!
At least no one tells us they could build Mito in a weekend :)
The Bar: Millions of Programmer Hours
Excel is certainly not the oldest piece of software, but it is likely the oldest end-user facing application with hundreds of millions of active users. Excel was first released in 1985, and has been under active development ever since.
The net result is that there are a literal ungodly number of programmer hours that have gone into building and maintaining Excel. The Excel team used to maintain their own C compiler, for god sakes (not sure if they still do).
While the three-person Mito team has some engineering talent (looking @ u teammates), we obviously don't have the capacity for most of this work. Mostly, we have to take shortcuts.
Excel has 450+ functions. Mito has the most popular 70 functions or so. Excel has about 100 keyboard shortcuts (the sign of a good analyst is never touching their mouse), Mito has an order of magnitude less.
Our product is full places where trying to match Excel's behavior would have resulted in another few months of developer time. That's fine if you've got 4 decades, hundreds of developers, and hundreds of millions of dollars. We're 0/3 on those, unfortunately.
Expecting the world, thinking it's table stakes
The result of the decades of polish that has gone into Excel:
- Excel defines what a spreadsheet is.
- Users doing the same task in Excel will take very different routes to get there.
Let me get concrete. One of the most popular use cases for automation within Mito is something we call "data recon". Generally, this process looks something like:
- Take some new data.
- Combine it with a "master data file."
- Label rows in this combined dataset.
We've seen users accomplish this task in Excel with:
- VLOOKUPs and IF statements
- Sorting and copy and paste and filtering
- Filtering, manual labeling, un-filtering, repeat
- Writing VBA code.
And this is all for just a simple data labeling task! One of the simpler use cases of a spreadsheet is actually twenty use cases in one.
Users show up to Mito expecting to be able to do things exactly the way they do things in Excel – and since Mito says "spreadsheet", of course this should just work. Excel defines what a spreadsheet is, after all!
The tens of millions of programmer hours that have gone into polishing Excel's functionality has given folks an infinite number of paths to complete their work. From what we've seen, there are users on every one of these paths.
When Mito lacks this level of feature polish, and doesn't support every possible workflow (e.g. VLOOKUPs are different, filtering has a slightly different execution model, VBA is not supported, etc), users get frustrated - why doesn't the damn spreadsheet just work how spreadsheets should work!
The Gravity of Excel
There are really only one things we can do to solve the above problems users run into: make Mito more similar to Excel.
In many ways, this is a useful and exciting direction. It makes it very clear what we're working towards, and it's pretty obviously exactly what our users expect us to do.
On the other hand, it raises a fair number of questions, especially as we're going to spend the next decade of our life on this tool. On the surface level, for what features is the gravity worth succuming to? More deeply, do we really want to just spend our time and talents replicating an existing applications? Hasn't UI and UX evolved a bit in the past few decades? Should we not try and do better?
There are no clear answers to these questions. What is clear is that the sharpest corners on our product currently are the places where we differ from Excel - and when rounding these corners, there's only one direction for us to go - down the gravity well of Excel.
Bang for buck optimization
I've painted a tough picture. Excel defines a spreadsheet, so users expect all of Excel's features in any spreadsheet, but it takes millions of programmers hours to replicate these features.
So, what do?
First, we have to prioritize ruthlessly and focus effectively. We cannot and will not make it if we try and handle every spreadsheet use case, as a single spreadsheet use case (like data recons) is actually twenty use cases in one. Right now, we're focused on data-checks - and we're going to polish the shit out of that flow.
Second, as an organization, developer velocity must be something we prioritize above all else. We must use systems thinking to figure out how to maximize our total throughput over the next few years. This includes large-scale metaprogramming, better dev scripts, more automated testing tools, and critically thinking about our personal productivity (e.g. learning to type faster!).
Third, we need to scale the team! No matter how well we prioritize, no matter how efficient we get, we're need more raw inputs. For a venture funded startup, we've been slower to scale than most. This is no longer sustainable, and if we're going to build a spreadsheet that is really excellent, we need to scale ASAP. Speaking of which, check out our job postings here!
More Like This
Automating Spreadsheets with Python 101
How to tell the difference between a good and bad Python automation target.
10 Mistakes To Look Out For When Transitioning from Excel To Python
10 Common Mistakes for new programmers transitioning from Excel to Python
Research shows Mito speeds up by 400%
We're always on the hunt for tools that improve our efficiency at work. Tools that let us accomplish more with less time, money, and resources.
3 Rules for Choosing Between SQL and Python
Analysts at the world's top banks are automating their manual Excel work so they can spend less time creating baseline reports, and more time building new analyses that push the company forward.