The TL;DR cloc Outputs
Before
----------------------------------------------------------------- Language files blank comment code ----------------------------------------------------------------- JavaScript 394 6882 906 29480 Less 69 1516 407 6690 YAML 2 56 0 222 Handlebars 5 6 0 60 JSON 1 0 0 10 ----------------------------------------------------------------- SUM: 471 8460 1313 36462 ----------------------------------------------------------------- Char count: 1337812
After
----------------------------------------------------------------- Language files blank comment code ----------------------------------------------------------------- Elm 81 2174 258 8993 JavaScript 26 475 182 1953 Sass 23 588 177 1761 CSS 1 2 1 264 HTML 1 3 0 43 ----------------------------------------------------------------- SUM: 132 3242 618 13014 ----------------------------------------------------------------- Char count: 452046
In case that’s not clear, the Elm version was able to express the same ideas in 35.7% of the lines of code. The original run was built by 2 developers in 15 months & the rewrite was done by essentially 2½ devs in 6 months. Now the UI is built on a strong static system & immutable data structures giving lawful guarantees along with boosting performance. That’s straight wins across the board.
Background
When I joined in on this project the highlights of the stack were React, Redux, Babel’d ES2015, Electron that worked only in Electron. That seemed like a pretty neat stack for 2015. But it had some glaring problems:
- A complete OO wrapper around Redux to make it stateful (which was undocumented & rendered Redux’s documentation useless)
- 2 additional methods of state management along side Redux
- Side-effects galore in smart components
- Marshalling of data looping it 3 different times before sending it to the view to be iterated once more
- A home-brewed caching layer in which “everything works, but the invalidation doesn’t always” (since that’s not a meme)
- Dependencies on VLC libraries to be compiled in
- A full Express REST interface that servered up data stored in a SQLite database that was populated from Node.js calls in Electron from another REST
- A number of dependencies made it Electron-only, not allowing it to see the light of day in a browser
- Some kooky code just to not introduce a utility library like Ramda or lodash (or anything that might be lawful)
-
Some of the abstractiest, thunkiest code with classes with methods like
this._init()which if you jumped to the definition 400 lines down you’d be greeted with_init() { this.init(); }…you know, just in case. “Just in case what?” I asked. “In case we need to add more (side-)effects on initialization.” (Side note: you may find this video by mpj a good explanation on understanding this pet peeve) - Deep, thick React flame charts that made the UI crawl
In Other Worlds
Another team had put together a bare-bones jQuery alternative for their debugging to prove that the project ran stand-alone in a browser, but also could be miles faster. This happened right around the time that we caught these huge show-stopping perf issues. A two-day skunkworks, like a scene out of Silicon Valley, between myself & two other workers provided a proof-of-concept that showed that we could in fact write a faster-rendering application that ran in a browser.
Crossroads
As a developer (& a human being), I’ve started to value my finite time on this 107,200 km/h-orbiting rock. The pieces of the code base I had implemented were heavily Elm inspired from my free-time projects & littered with point-free Ramda (which I gave a talk about in Loveland, Colorado). However, the core of the system was so OO-wrapped, & the performance so bad partially due to the base React & iteration choices, that I felt very strongly that one of two things needed to occur: either we do a rewrite to crush some of these architecture issues or months of refactoring that wasn’t going to lead to many new gains outside of performance. If the latter had happened, I likely would have jumped ship since I have little desire in this lifetime to lipstick any OO pig.
At this point, I let the CTO know about my feelings on the matter—& how I would seriously like to rewrite the application from the ground up, faster, cruftless, & with guarantees with my side-project-go-to language, Elm. I got okayed.
First Time Leading & Architecting
Honestly, I’d never had a shot to greenfield a project & make big decisions like libraries & time allocation. So after talking with my coworkers (& finally being taken seriously O-o) to address the problems we went with these ideas:
- Replace React + Redux + home-brewed OO wrapper with Elm
- Replace the Electron Node.js polling & then using Express REST interface for data retrieval with a polling client-side Web Worker & IndexedDB for offline storage
- Marshalling is done twice—once at the after the REST call which is stored in the exact manner we want it to be sent thru the Elm port on-demand & the second is pulling it thru the port & converting integer enums to types, folding on device data, & other things for the view
- Replace grunt+gulp+browserify with Webpack
- Drop the Babel dependency in favor of supporting modern, evergreen browsers
- Drop that VLC dependency in favor of HLS via hls.js
- Remove as much state & side-effects as possible
- Be reasonably point-free
- Stick to libraries instead of frameworks
- Delegate the ‘back-end’ work of REST retrieval & offline storage to a coworker that knows JS & understands NoSQL-style databases & the REST API much more than myself
- Fix one of the biggest performance issues with infinite scrolling of lists & tiles by contracting that piece out to my homeboy Isaac Shapira since he’s smarter than me :P
So What Was It Like?
Broad-stroking that question: pleasant—the best experience I’ve had writing code (professionally I’ve done JavaScript, PHP, Python, ClojureScript). I would honestly say that my favorite experience was when my boss would ask me something along the lines of “what’s it going to be like to implement this change?”, & my response was always “well, it’s going to take a bit of prep to define the problem, choose the best data structure, & getting the new stuff to compile, but I promise you: once it compiles, it will run.” And, man, that was a true statement.
At our first QA after writing nearly 8000 lines of Elm, I got 2, yes 2, bugs on the front-end.
Both bugs were in my logic & took less than 2 hours to fix.
The second round of QA, there was 1 bug in some layer of the JS port code.
If only we had had some time to write like 10 tests we could have caught the first two… the rest was all assurances gained by Elm’s type system.
Never have I had so few bugs considering the amount of code being tested—and I didn’t even get to write those tests.
A strong type system renders most unit tests useless the managed side-effects & single source of truth state, meant we could just expect things to work.
There were no runtime errors.
We had had a bad package roll thru on npm which broke the build, but you better bet that Elm’s semantic-version-enforcing package manager never failed.
The shallow flame charts of Elm’s virtual DOM (VDOM) implementation made it much easier to diagnose performance hiccups (altho I still want to get better at these diagnostics).
The code used basically every *-extra package in the system since nice functions like unwrap & singleton are ‘hiding’ here.
The code density was super high & concise expressing most of the ap`p in terms of ``map` & fold.
The code tries to be ‘lawful’ (in the Haskell sense) with use of monads & applicative functors as well as optics, with Return & Monocle.
What Were the Pain Points?
- Webpack pretty much blows—it’s configurations are a mystery & it was leading to segfaults + errors about too many files open which ended up outputting a critical failed build that didn’t actually fail in Jenkins. I would love to rig up Tup or Shake & devd or something similar instead. (elm-webpack-loader has since been ehanced to deal with some of this, altho I still think Webpack has too much voodoo)
-
The standard library lacked some of the most useful functions which is why I had to import all of those
*-extrapackages—particularly, many of the libraries include functions for the Functor & Monad instances of a “type”,map&andThen, but not the Applicative functionssingleton&andMap. -
It would have been nice to have some sort of reach-into-able global state like Haskell’s State Monad for everything related to internationalization (i18n). In JS land, you’d partially apply in your language into the
Intlcollators or in the translate function I had talked about in a previous blog post, attach it to something globally, & just conveniently accessing those functions in the view where needed instead of having to thread a record of i18n functions thru all layers of the views. There’s niceness in not having more global state at the cost of convience in this case. -
Union types not being
comparableis really annoying. -
The lack of type classes really hosed a last-minute, main-data-structure-changing requirement (the project could have been out in 5 months). Rather than letting
mapbe polymorphic, I had 8000 lines to go thru & reassess with the new structure. The compiler helped guide me, but I, personally, would have preferred just being more polymorphic. This also was the main reason lenses were used (the other being the how loud & nasty the update function would get). (Edit: after talking with Feldman it turns out there was a pretty big misunderstanding due to the words in the documentation around ports + Json.Value not having to be JSON—concerned with the performance hit for stringifying just to send in to the port, I went with anEitherdata structure that ended up being less than ideal & at times rather complicated). - The lack of Rank-N types tripped us up a couple of times.
- Certain web APIs & features just aren’t covered by Elm yet, & some structures & libraries from more mature languages are missing—but those should come with time.
In Summary
Honestly, it was the most sound piece of software I’ve helped write & lead. It really is as I had expressed in the talks I had given: it’s like Redux + React + Ramda + Flow + etc. all rolled into a language that was better than bolting on to the JavaScript runtime & trying to piece together & learn each of those parts. You can express big ideas in very little code—unlike some of the niftier type-classed options like PureScript—Elm is very approachable to non-FP teams looking to transition. 35.7% the code size is no joke; that is a lot less code to maintain & it feels good writing it. We all know that place in the code base labeled “Here Be Dragons”, but with the type system & compiler having your back, it wasn’t scary to do a lot of (constant) refactoring—especially as we learned more advanced functional programming “techniques”. I could not recommend more folks that have shoddy React apps or folks looking to leave the Angular train to do Elm instead. You can ramp up an OO developer or front-end-inexperienced developer as fast if not faster than React et al. for these reasons. I promise it’s less of a learning curve than learning a new framework.