The TL;DR cloc Outputs

Before:

-----------------------------------------------------------------
Language       files          blank        comment           code
-----------------------------------------------------------------
JavaScript       394           6882            906          29480
LESS              69           1516            407           6690
YAML               2             56              0            222
Handlebars         5              6              0             60
JSON               1              0              0             10
-----------------------------------------------------------------
SUM:             471           8460           1313          36462
-----------------------------------------------------------------
Char count: 1337812

After:

-----------------------------------------------------------------
Language       files          blank        comment           code
-----------------------------------------------------------------
Elm               81           2174            258           8993
JavaScript        26            475            182           1953
Sass              23            588            177           1761
CSS                1              2              1            264
HTML               1              3              0             43
-----------------------------------------------------------------
SUM:             132           3242            618          13014
-----------------------------------------------------------------
Char count: 452046

In case that’s not clear, the Elm version was able to express the same ideas in 35.7% of the lines of code. The original run was built by 2 developers in 15 months and the rewrite was done by essentially 2½ devs in 6 months. Now the UI is built on a strong static system and immutable data structures giving lawful guarantees along with boosting performance. That’s straight wins across the board.

Background

When I joined in on this project the highlights of the stack were React, Redux, Babel’d ES6, Electron that worked only in Electron. That seemed like a pretty neat stack for 2015. But it had some glaring problems:

• A complete OO wrapper around Redux to make it stateful (which was undocumented and rendered Redux’s documentation useless)
• 2 additional methods of state management along side Redux
• Side-effects galore in smart components
• Marshalling of data looping it 3 different times before sending it to the view to be iterated once more
• A home-brewed caching layer in which “everything works, but the invalidation doesn’t always” (because that’s not a meme)
• Dependencies on VLC libraries to be compiled in
• A full Express REST interface that servered up data stored in a SQLite database that was populated from Node.js calls in Electron from another REST
• A number of dependencies made it Electron-only, not allowing it to see the light of day in a browser
• Some kooky code just to not introduce a utility library like Ramda or lodash (or anything that might be lawful)
• Some of the abstractiest, thunkiest code with classes with methods like this._init() which if you jumped to the definition 400 lines down you’d be greeted with _init() { this.init(); } …you know, just in case. “Just in case what?” I asked. “In case we need to add more side-effects on initialization.” (Side note: watch this video by mpj to understand my pet peeve)
• Deep, thick React flame charts that made the UI crawl

In Other Worlds

Another team had put together a bare-bones jQuery alternative for their debugging to prove that the project ran stand-alone in a browser, but also could be miles faster. This happened right around the time that we caught these huge show-stopping perf issues. A two-day skunkworks, like a scene out of Silicon Valley, between myself and two other workers provided a proof-of-concept that showed that we could in fact write a faster-rendering application that ran in a browser.

Crossroads

As a developer (and a human being), I’ve started to value my finite time on this 107,200 km/h-orbiting rock. The pieces of the code base I had implemented were heavily Elm inspired from my free-time projects and littered with point-free Ramda (which I gave a talk about in Loveland, CO). However, the core of the system was so OO-wrapped, and the performance so bad partially because of base React and iteration choices, that I felt very strongly that one of two things needed to occur: either we do a rewrite to crush some of these architecture issues or months of refactoring (which I had coined “unfuctoring”) that wasn’t going to lead to many new gains outside of performance. If the latter had happened, I likely would have jumped ship since I have little desire in this lifetime to lipstick any OO pig.

At this point, I let the CTO know about my feelings on the matter—and how I would seriously like to rewrite the application from the ground up, faster, cruftless, and with guarantees with my side-project-go-to language, Elm. I got okayed.

First Time Leading and Architecting

Honestly, I’d never had a shot to greenfield a project and make big decisions like libraries and time allocation. So after talking with my coworkers (and finally being taken seriously O-o) to address the problems we went with these ideas:

• Replace React + Redux + home-brewed OO wrapper with Elm
• Replace the Electron Node.js polling and then using Express REST interface for data retrieval with a polling client-side Web Worker and IndexedDB for offline storage
• Marshalling is done twice—once at the after the REST call which is stored in the exact manner we want it to be sent through the Elm port on-demand and the second is pulling it through the port and converting integer enums to types, folding on device data, and other things for the view
• Replace grunt+gulp+browserify with Webpack
• Drop the Babel dependency in favor of supporting modern, evergreen browsers
• Drop that VLC dependency in favor of HLS via hls.js
• Remove as much state and side-effects as possible
• Be (reasonably) point-free
• Stick to libraries instead of frameworks
• Delegate the ‘back-end’ work of REST retrieval and offline storage to a coworker that knows JS and understands NoSQL-style databases and the RESTAPI much more than myself
• Fix one of the biggest performance issues with infinite scrolling of lists and tiles by contracting that piece out to my homeboy @fresheyeball since he’s smarter than me :P

So What Was It Like?

Broad-stroking that question: pleasant—the best experience I’ve had writing code (professionally I’ve done JavaScript, PHP, Python, ClojureScript). I would honestly say that my favorite experience was when my boss would ask me something along the lines of “what’s it going to be like to implement this change?”, and my response was always “well, it’s going to take a bit of prep to define the problem, choose the best data structure, and getting the new stuff to compile, but I promise you: once it compiles, it will run.” And, man, that was a true statement.

At our first QA after writing nearly 8000 lines of Elm, I got 2, yes 2, bugs on the front-end. Both bugs were in my logic and took less than 2 hours to fix. The second round of QA, there was 1 bug in some layer of the JSport code. If only we had had some time to write like 10 tests we could have caught the first two… the rest was all assurances gained by Elm’s type system. Never have I had so few bugs considering the amount of code being tested—and I didn’t even get to write those tests. A strong type system renders most unit tests useless the managed side-effects and single source of truth state, meant we could just expect things to work. There were no runtime errors.

We had had a bad package roll through on npm which broke the build, but you better bet that Elm’s semantic-version-enforcing package manager never failed.

The shallow flame charts of Elm’s virtual DOM implementation made it much easier to diagnose performance hiccups (although I still want to get better at these diagnostics).

The code used basically every *-extra package in the system because of the nice functions like unwrap and singleton. The code density was super high and concise expressing most of the app in terms of map and fold. The code tries to be lawful with use of monads and applicative functors as well as optics, with Return and Monocle.

What Were the Pain Points?

• Webpack pretty much blows—it’s configurations are a mystery and it was leading to segfaults and errors about too many files open which ended up outputting a critical failed build that didn’t actually fail in Jenkins. I would love to rig up Tup or Shake and devd or something similar instead. (elm-webpack-loader has since been enhanced to deal with some of this, although I still think Webpack has too much voodoo)
• The standard library lacked some of the most useful functions which is why I had to import all of those *-extra packages—particularly, many of the libraries include functions for the Functor and Monad instances of a “type”, map & andThen, but not the Applicative functions singleton & andMap.
• It would have been nice to have some sort of reach-into-able global state like Haskell’s State Monad for everything related to internationalization (i18n). In JS land, you’d partially apply in your language into the Intl collators or in the translate function I had talked about in a previous blog post, attach it to something globally, and just conveniently accessing those functions in the view where needed instead of having to thread a record of i18n functions through all layers of the views. There’s niceness in not having more global state at the cost of convience in this case.
• Union types not being comparable is really annoying.
• The lack of type classes really hosed a last-minute, main-data-structure-changing requirement (the project could have been out in 5 months). Rather than letting map be polymorphic, I had 8000 lines to go through and reassess with the new structure. The compiler helped guide me, but I, personally, would have preferred just being more polymorphic. This also was the main reason lenses were used (the other being the how loud and nasty the update function would get). (Edit: after talking with Feldman it turns out there was a pretty big misunderstanding due to the words in the documentation around ports + Json.Value not having to be JSON—concerned with the performance hit for stringifying just to send in to the port, I went with an Either data structure that ended up being less than ideal and at times rather complicated).
• The lack of Rank-N tripped us up a couple of times.
• Certain web APIs and features just aren’t covered by Elm yet and some structures and libraries from more mature languages are missing—but those should come with time.

In Summation

Honestly, it was the most sound piece of software I’ve helped write and lead. It really is as I had expressed in the talks I had given: it’s like Redux + React + Ramda + Flow + etc. all rolled into a language that was better than bolting on to the JavaScript runtime and trying to piece together and learn each of those parts. You can express big ideas in very little code—unlike some of the niftier type-classed options like PureScript—Elm is very approachable to non-FP teams looking to transition. 35.7% the code size is no joke; that is a lot less code to maintain and it feels good writing it. We all know that place in the code base labeled “Here Be Dragons”, but with the type system and compiler having your back, it wasn’t scary to do a lot of (constant) refactoring—especially as we learned more advanced functional programming “techniques”. I could not recommend more people that have shoddy React apps or people looking to leave the Angular train to do Elm instead. You can ramp up an OO developer or front-end-inexperienced developer as fast if not faster than React et al. for these reasons. I promise it’s less of a learning curve than learning a new framework.

I don’t have a comments section, but if you have any specific questions, feel free to ask on Reddit: https://www.reddit.com/r/elm/comments/5fy1hr/rewriting_react_releasing_elm_into_the_wild_a/