Over the past few months, I have been working with Meteor.js, henceforth called Meteor, building out some complex UX workflows. Being a platform architect and systems developer, I had shied away from working on product UI’s. However, sitting in innumerable meetings discussing the urgency of UI development and the difficulty of hiring good UI developers, I decided to jump in and contribute to our team’s UI effort. I was emboldened by my recent research into Meteor and its potential capabilities in comparison to Angular, React, Ember, Derby and other sub-frameworks if you will. Herein, I summarize my experience and summarize some key lessons as my Meteor journey continues.
As I started getting into the amorphous world of UI development jargon with template languages, HTML V, different javascript libraries, AJAX calls, asynchrony hell etc. there were a number of daunting subjects on top of my mind as a server side developer:
- HTML, HTML V, CSS related – What are the variations ? What can I do purely with HTML out-of-the-box ? What are the CSS issues ? What are the clean abstractions ? What should be in HTML, what in Javascript and what in CSS ? How do we deal with mobile and responsive design issues? How do we deal with browser-specific issues ?
- Javascript for the JAVA developer – Writing event-driven functional programs that run in a single thread is a bit different from writing conventional multi-threaded programs. Organizing the code-base, handling asynchrony via events, understanding how to control the flow of events such that the appropriate UI behavior is exhibited has to be done quite a bit differently then how one would do on the server side.
- MVC babel – Every UI framework talks about model-view-controller with agreement only on the “model” but varying notions of where the view and controller code execute, and how they are triggered. For a server side developer – the pattern of observers and observables is quite well known – and one wonders why all this confusion. Terminology such as data binding, controllers in different bits of code and server-side patterns being adopted for UI development leading to confusion.
- Disquiet with the loose, hacky nature of it all – Much of the doubt comes from i) Javascript – interpreted functional code – with no typing, complexity of the event models, prototypical objects (rather than classical java or C++ style OO), scoping – lexical versus dynamic, global versus local vars ii) the build process – how many javascript files should I include, where ? How do I package files, in what order ? How do I minify? iii) What goes on the client and what goes on the server side? Much of this is also exacerbated with implementation of REST APIs that differ in quality and abstractions.
- UI development cycle – How do I work with my design and HTML team ? Who does what on which piece ? How do we pull these different pieces together in every change cycle and feature iteration ? Folks with different backgrounds and deliverables needed to work on a tightly coupled artifact on tight deadlines with little slack for redo efforts. What are the dependencies on the code-base?
- Building mobile front ends – Considering the amount of effort sunk into developing a good browser-centric UI, how does one transition relevant bits of the effort to a mobile experience ? What elements changes and what do not ? What is better done as a “native UI” experience? This is key for startups building out products – we need good enough front-ends in the shortest time-frame that encapsulate the product vision and experience to enable product testing and UX feedback ( See the recent article on Google Inbox development)
So how does/did adopting Meteor help in ameliorating some of the above issues:
- Meteor provides a clean way to decouple HTML, CSS and JS and handles all the file dependencies. Further with Blaze (the templating language) and the notion of events and helpers it helps a backend developer quickly get a user interface up and running quickly. The javascript dependencies are resolved when everything is compiled (though the depth-first priority order of files may be bit confusing). A key aspect is the ability to store “state” inside Mongo and manage it effectively without worrying about how to access user state on client.
- Meteor also allows handling the javascript event model with a clean abstraction for linking synchronous and asynchronous calls on both server-side and client-side. The AJAX model can be simplified in a clean manner by constraining all out-going calls to be from the Meteor server side rather than directly from the browser. This allows for better development as the event-model can be debugged properly by constraining the state-evolution to occur only at one place – the underlying mongo collections.
- The MVC model can be mapped in a clean manner – Model in the Mongo state, View in the Blaze templates, “Controller” – managed explicitly via state updates on collections (followed by) the implicit reactive updates. The state updates can happen synchronously or asynchronously and performed either by the user or by backend calls. This permits a cleaner conceptual analysis as UI flows get more complex. However, understanding reactivity is quite important (see more below).
- Meteor handles all the packaging and allows development of Single-Page Applications (SPAs) in a clean manner. Furthermore, it allows easy inclusion of third-party JS libraries, which provide a lot of functionally out-of-the-box. All the major libraries – jquery and others are supported out of the box. Also, it handles scoping variables and provides some basic design patterns (for the first time JS developer) to get going and build a functional UI without all the major pitfalls of generic JS development.
- By architecting the subsystems of the Meteor applications appropriately, it is possible for the a) app designer – who works on the HTML templates and CSS, b) the UI Javascript developer, c) the Server-side JS/API developer and d) the Mongo/DB developer to work in “concert” to build out a feature all the way from UI through the backend – on the same code-base in an iterative, incremental manner. The different subsystems may run in different places on the network but every person works on the feature in a coordinated manner. This has enabled us to be more productive in a short time with minimal hand-offs between team members. Furthermore, with the low learning curve (assuming you have not picked up other mental models of javascript development), team members learnt quickly as they could experiment and iterate quickly.
- Finally, as Meteor continues to evolve and with the recent integration of the Cordova toolset for mobile app development, there is a clean path to deploy and test your product on mobile devices. So from a product management perspective, if you architect the product properly, one can build for all form-factors and get into the test cycle early. UI quality out-of-the box is pretty good and the code-base is clean for the team to manage. Further supporting both android and iOS, it is a big time-saver from a product management perspective.
However, even with these benefits, there are a number of common pitfalls. Some of which I have run into and also gleaned from conversations with developers who have explored meteor after cutting their teeth in other frameworks.
a) Handling reactivity in Meteor – Understanding reactivity is key. When and how does it get activated ? What are the basic “reactive” variables out of the box (user, Session, Mongo collections) and how do they impact your app design.
b) Moving large amounts of data between the client and serverside – Should you pull it into Mongo first and then subscribe on client or vice-versa? Should one even use Minimongo ? What are its limitations?
c) Using Subscriptions – How to effectively use subscriptions ? Subscriptions allow you to pull data into the client on demand (this minimizing how much data you have on your client). However, you also need to understand how to keep collections separate or share on minimongo. A single collection can power different UI views so when routes change, subscriptions also are reset. Much of this knowledge is evolving actively as developers use Meteor in different contexts.
d) Routing – IronRouter is the default router but behaves a bit differently from other off-the-shelf routers. It is a reactive router and affects subscriptions and other reactive parameters. Utilizing this along with session variables is key to building effective navigation in your app. Errors here can be costly for the app workflows.
e) Using Jquery and the associated event model – Using jquery patterns to locate elements and update element states has to be carefully thought through. Understanding of the interplay of Jquery and rendering updates is still evolving in the context of template.rendered callbacks.
f) Organizing Blaze templates – Blaze templates can be organized granularly to make effective use of reactivity. Isolating different levels of updates of the UI can considerably speed up your app. This requires a bit of iterative template re-factoring and understanding your data to get it right.
Though Meteor considerably speeds up the UI implementation process, there are some key design choices that the UI architect has to make. Design choices include:
- The shared collections between client and serverside and defining subscriptions/pubs to coordinate data flow (over the DDP protocol).
- Handling session management in the app (with or without cookies).
- Template layouts
- Router Layouts and Use of Session variables to activate template renderings
- Event models and standardizing on event behaviors
Having Meteor in the toolkit of a product manager (at a startup especially or otherwise) – enables you to think differently about how to build, deploy and test your product to get the early feedback. Many functionalities that may be considered back-end can be pushed into the Meteor layer and managed. Features can be added incrementally as user feedback is gathered and the product experience is fleshed out before scaling out. Meteor also enables incremental API development as you understand the units of “granularity” one may want to support. The basic meteor out-of-the-box can support testing on small audiences most of the core functionality of a product. Furthermore, much of this can be built out with small teams. From an overall perspective – either strategy from product to platform or platform to product can be effectively executed as you manage the “functional” footprint of the product – building the MVP to get traction. Furthermore, using Meteor does not preclude the use of Angular or React (two popular frameworks) – they can still be used within the Meteor framework in an incremental manner. So you start developing your product with least technological commitment and can evolve as your product-market fit evolves. In future blogs, I will share some insights into using Meteor to power data-centric versus workflow versus realtime apps.