Henrik Joreteg

Follow this blog

Project Fugu: A New Hope

Capabilities. That's what The Web needs. Things like: Face or other biometric logins for websites USB connectivity TCP and UDP sockets clients / servers Serial connections Geofencing and background location tracking Screen lock / screen brightness Barcode scanning / generation Contacts API. etc., etc., etc. If a native app can do it, the web should be able to do also! Progress! As it turns out the folks on the Chrome team feel the same. In fact, there's an entire public spreadsheet with a really ambitious list of capabilities that have been identified as gaps between what the web can do and what native apps can do. The web as an application platform, has an incredible future. See it for yourself! goo.gle/fugu-api-tracker This gives me a new hope for the future of The Web! Meanwhile over on IE, ahem... I mean Safari The following appears to be the general attitude I see from the Webkit folks even for adding something as benign as screen lock. Ricky is a WebKit engineer and @othermaciej who responds with a similar attitude is currently head of WebKit Engineering: Looking forward to the battery life benefits this API will bring. — othermaciej (@othermaciej) September 12, 2019 That makes me sad. Why does any of this matter? In short, because software distribution is hard. Every other means of distribution involves a variety of B.S. that is unique to each platform you want to distribute to. This involves, but it not limited to: Building/packaging Code signing App store approval seeking App store revenue sharing. Risk of being demoted in the lottery of app store picks and search engines. Dealing with obsolescence due to platforms moving forward without backward compatibility. Every time you want to do a material update, you have to repeat this same silly song and dance again... for each platform. And most of these little steps and requirements are different for every platform you want to distribute to. Take Windows, for example. If you want to provide a link to download an electron app installer from your site and you don't want to have to coach your users how to click through the Windows warnings when they try to run the installer you have to do code signing. Fine. Ok. No big deal, right? Well, I'm doing this right now so... you have to: Buy an Extended Validation cert for \$349 (a year) Complete some really extensive information about your business. Submit/upload a bunch of documentation, including a formal attestation letter from your CPA (who obviously won't do this for free) Then, once everything is submitted, they will mail you a USB drive with the certificate on it. Then you get to sign your software. Doable? Sure. Fun? Definitely not. Cost effective and sustainable? Not really. So, why not just stick to the web!? That's what I'm trying to do! But sometimes, you need to do something that the web does not yet allow. More frequently, this is becoming "you need to do something that Apple doesn't think the web should be able to do" (or won't prioritize the work required to implement). This happens both in my consulting work (currently with Walmart, previously at Starbucks), and in my AnesthesiaCharting.com business. We keep running into these limitations of what the browser can do. The common answer is, well... so just build native apps for that, or use Electron. Ok, let's look a little closer, maybe it's not actually so bad? If you really "go all in" on native, you have to build at least three different code bases instead of one. If you're like my big clients, maybe that's worthwhile... maybe. But that doesn't mean it isn't a a lot more work. So, now you're probably thinking: "Just use React Native, or Flutter, or Electron or some other meta platform!" But that still doesn't solve it, for several reasons. For example let's look at my business AnesthesiaCharting.com. We'd ideally want to support the following platforms: Windows Desktop Mac Desktop iPad/iOS Android Phones/Tablets Chromebooks So, I guess React Native can now mostly support all these. Flutter has a "alpha" support for Mac OS but not Windows. And there's a giant gap in all these ideas: none of the problems outlined previously about distributing on multiple platforms goes away. If you actually want to distribute and update software to all these platforms you still have to get approval everywhere, pay your developer fees, so you can sign your code, and set up a big complicated release process for pushing updates. I feel like most people that throw this answer out have not actually done this themselves. It seems they're frequently spouting off things that they've heard at conferences from what I can only call "the marketing department" for these various technologies. What do all those platforms have in common? All but one of those 5 platforms I listed above can run Chrome! That means that I have a very capable web runtime available on all but one of the platforms that my customers use. Oh, in case you're wondering--iOS cannot run Chrome. Don't be fooled by the fact that you can install an app called "Chrome" from the Apple app store. It is not actually Chrome, in fact, it's not even Safari. It's less than Safari. It's a WebKit web-view which, cannot even have things that Safari supports like ServiceWorker, let alone any of the other, more advanced APIs that expose the capabilities of the device. So what specific capabilities do I need that Web doesn't yet provide? I've been finding myself working with serial ports recently. Yeah, that's right, good 'ol RS-232. A standard for device to device communication first published in 1960! It's dinosaur tech, but unlike the big lizards, this tech has stuck around. As it turns out, one of the most popular patient vitals monitors (you know the ones that show your heart rate, oxygen saturation, etc) is made by a company called Criticare. These things you can only talk to via, you guessed it: serial. But it's not just for old tech, lots of robotics stuff and other simple controllers use serial connections. As it turns out Google is working on a Web Serial API right now! It's in the Fugu tracker! In fact, it's already shipped as an Origin Trial that I'm in... and I'm stoked! All I have to do is: const port = await navigator.serial.requestPort(); await port.open({ baudrate: 38400 }); Now I can read and write to the connected device! Why is that a big deal? With those two lines above. I can distribute software via The Web that talks to these patient vitals monitors. And it works with no additional work on 4 of the 5 platforms I'm trying to target! All I have to do is tell customers to use a Chromium-based browser. Pedantry You can be as pedantic as you want about "browser engine diversity" you can say that "Google has too much control of the web", you can argue about all manner of pedantic things. But at the end of the day, as a business trying to ship software to my customers, guess what I'm going to do? I'm going to tell my customers to use a Chromium-based browser if they need to talk to a Criticare monitor. I'll tell them to skip the fancy iPad Pro and buy a \$300 chromebook to use as a clinical tool in their operatories. Forget Apple. I'm going to tell all these doctors that Apple is selling them short and to buy a cheap chromebook or other device that can run a Chromium based browser. Oh and hey, before you get too pedantic on me. This is a reminder that Microsoft is taking Chromium and using it to compete with Chrome on user-facing features! So is Brave and others. This is The Web we want. A Web with all the capabilities we need, while keeping software distribution SIMPLE. Let the vendors compete on user-facing features, like password managers, privacy controls, etc. But, let's have the most capable open source browser engine available so we can get the capabilities we need to ship software. This is why Project Fugu give me a new hope. Thanks for reading! See ya on Twitter, I'm @HenrikJoreteg on there if you want to kindly tell me all the ways I'm wrong :)

Improving Redux state transfer performance with JSON.parse(), a quick case study

TLDR: Turning Redux state into a JavaScript string you can parse with JSON.parse() instead of an object literal, or inert script tag, appears to be significantly faster than other approaches for sending Redux store state to the browser. For my case making this one change shaved TTI (Time To Interactive) from an already pretty good 4.04s to 3.3s, a .74s or ~18% improvement. It increased the Lighthouse performance score an average of 8 points from 87.2 to 95.2! These results were unexpected, when I shared them on Twitter both Addy Osmani and Mathias Bynens asked me write it up... so here we are. Introduction When doing Server-side Rendering (a.k.a SSR) it is common practice to render HTML and "rehydrate" it in the browser using Preact or React. Often times, this also requires you transfer the current application state to the browser, somehow. That way, when your application spins up in the browser, it can rehydrate with all the state it had on the server. Most of my experience with doing this as a consultant is with Redux. And, in my experience these Redux state dumps can easily get way too big... but I digress. The best strategy is always to minimize their size to begin with. But, there are certain cases where that is difficult to do beyond a certain point, and plenty of other cases where things have already gotten out of hand and you're just trying to make thing better. There are three predominant mechanisms (that I'm aware of) for doing this "state transfer." They are: Option #1: Turn your state in to a JS object literal This means that on the server-side you do something like JSON.stringify(store.getState()) and you end up with HTML that includes a script tag like this. <script> window.__REDUX__ = { some: "value", someOther: "value and so on" }; </script> For this approach to work you also have to be careful about not including anything in your JSON that would make the browser think the script tag was terminating. So some extra escaping may be in order (learn more here). Additionally, using the result of JSON.stringify() directly in JS source code was a potential security issue until recently, since JSON strings could contain certain characters that were invalid in JS strings. More recently, this has been addressed with a spec-level proposal that made it into ES2019. But, as you'll see you probably won't want to use this approach anyway since it's the slowest option. When this executes in the browser what was JSON on the server-side is treated and parsed by the browser as a JS object literal. No different than if you have written code like this, without the quoted key names: <script> window.__REDUX__ = { some: "value", someOther: "value and so on" }; </script> But, it just has a few extra double quotes. The important point in this case is that the data itself is parsed as if it were JavaScript. Option #2: Turn your state into an "inert" script Browsers only will parse the contents of a <script> tag if its type is "module", "text/javascript", or another recognized JS MIME type, or well... it doesn't have a type specified at all. So the point is anything else you specify as a type means its contents will be ignored. People use this for all manner of trickery, but it can also be used to transfer Redux state with a bit less escaping, but again, you still need to escape for stuff like </script to protect against XSS issues. Some people like to to give it a valid MIME type to make themselves feel better but... browser don't care. For example: <script id="myState" type="application/json"> {"your": "state", "goes": "here"} </script> Then you can later parse it by grabbing the element out of the DOM and parsing its contents in some other script like so: window.__REDUX__ = JSON.parse( querySelector("myState").innerHTML ); Unlike approach #1, in this case, the browser never interprets the data as JavaScript it merely parses a string as JSON, which is a much stricter data type, which as it turns out, is much faster. However, this approach means that the browser first has to create a DOM element with a lot of text in it and then you have to read that text from the DOM. This, as it turns out, is not without performance overhead of its own (results below). Option #3: Turn your state into JS String containing escaped JSON. What you do end up with in your HTML is this: <script> window.__REDUX__ = JSON.parse("{\"some\":\"state\",\"other\":\"value\"}"); </script> As it turns out, this is way faster than the other options. It may look a bit funny, but you still end up with a JS object. However, the key difference is that the contents of the data you're sending is never interpreted as if it were code. It's just a long JavaScript string literal, as far as the JavaScript parser is concerned. But, at runtime, it's parsed as JSON and becomes a JS object instance just like all the other approaches. A quick note on how I generated the JSON string: It turns out, it's not so easy to do this correctly, especially if you're including data from an API that you may not have full control over. Avoiding XSS is important here. Anyway, again thanks to Mathias, I ended up doing the following in node.js on the server-side to generate that string in a way that's safe to assume can be treated like a JavaScript string. I'm using his excellent: jsesc library as well as his advice to do roughly something like this: const jsesc = require("jsesc"); // assume `data` here is the data we want to transfer module.exports = data => { const jsonString = jsesc(JSON.stringify(data), { json: true, isScriptContext: true }); return ` <!DOCTYPE html> <html> <head> ... stuff here </head> <body> <div id='js-app'>${allTheAppHTML}</div> <script>window.__REDUX_STATE__ = JSON.parse(${jsonString})</script> <script src='/my-app.js'></script> </body> </html> `; }; I'm a stooge, all the credit goes to other people here The inimitable Addy Osmani wrote an awesome post for the V8 project's blog about The Cost of JavaScript in 2019. And I happened to see, the equally inimitable, Mathias Bynens tweet where he screencapped part of it about the cost of parsing JSON and suggested the JSON string approach (and later helped review this post). I'm currently working on a Preact PWA project for a big client where we're doing SSR with a redux state transfer of an object bigger than I'd like it to be, so... i figured what the heck, let's try it. What I found blew me away. The test setup Browser: Chrome Stable 75.0.3770.100 Audit Mechanism: Lighthouse Perf Audit, incognito mode, Simulated Fast 3G, 4x CPU Slowdown setting, with no caching. Number of tests: 5 runs of each approach Size of Redux state in bytes: 163,980 with escape slashes: 176,017 added 7% The size of the state object here is significant. It's much bigger than I'd like it to be but it comes from an API that I have little control over so anyway... The results Approach #1: JS Object Literal Perf Score TTI 83 4.0s 87 4.1s 89 4.1s 88 4.1s 89 3.9s Avg: 87.2 4.04s Approach #2: Inert Script Tag Perf Score TTI 90 4.0s 90 4.0s 92 4.1s 92 4.0s 90 4.0s Avg: 90.8 4.02s Approach #3: Escaped JSON String inside Script Tag Perf Score TTI 95 3.3s 95 3.2s 96 3.3s 96 3.4s 94 3.3s Avg: 95.2 3.3s Summary Avg Perf Score Avg. TTI Approach #1: 87.2 4.04s Approach #2: 90.8 4.02s Approach #3: 95.2 3.3s Conclusion: JSON String inside a real <script> tag wins by a shocking margin. An 18% TTI improvement on an already pretty fast app is incredible for such a simple change. Your mileage may vary, but if you're doing a state transfer like this I'd encourage you to try it for yourself. Please tell me what you learn, I'm @HenrikJoreteg on Twitter. Please, let me know what you find. Some of my open questions that I don't really have time to dig into right this minute: As Mathias pointed out to me in DM Chrome 76 includes even faster JSON.parse, how will this perform in Chrome 76+? Could this be even better?!?! Would .innerHTML versus .textContent make any difference in approach #2? How does this change impact performance in other browsers? I'd love to hear your thoughts, but I wanted to share this while it was all still fresh in my mind. Hope this helps someone, thanks for reading!

Architecting UIs for Change

Techniques for keeping your app code under control even when requirements shift dramatically. I've had this pinned on my twitter for just about as long as "tweet pinning" has been a thing: If you don't actively fight for simplicity in software, complexity will win... and it will suck. It's a nice, pithy little quip, perhaps... but how do you actually accomplish this? The way I see it, broadly speaking, you have three options: Push back on any complex requirements. Just avoid the complex problems altogether by only building relatively simple things. Minimize entropy by trying to isolate areas of your app into distinct concerns and do whatever you can to hold those lines. Develop coping mechanisms for the inherent complexity we're bound to face and learn to manage it. Option #1 is untenable and frankly—boring. Yet, there are a surprising number of folks who seem to believe the web should still just be a set of linked static documents of content. But, some of us like to build more interesting things on the web that are truly app experiences. Not just documents, but document editors. Oh, and things like video editing suites, graphic design tools, interactive sheet music for learning how to play guitar and tools for tracking and monitoring patients during surgery. Option #2 is well-intentioned and seems reasonable on the surface. I've heard this desire expressed from management from the last two big corporate PWAs I've worked on. There's a logical desire to have distinct teams that are able to develop, build, and deploy in isolation from one another. To be clear, I'm all for separation of concerns. But the entire presentation layer of a nice cohesive "app" experience is, in fact, a shared concern and not something that can be divided so simply (more on this later). Options #3 As you probably guessed, this is my preference and will the subject of this post. But, before we can develop coping mechanisms, we have to study the problem. Perhaps we can discern something about the source of complexity. Identifying the source of complexity If you're a developer, I bet you're a little bit like me, you have an internal barometer that can sense how in-control the codebase is at any one time. I feel a sense of dread as we start to pile on features if I'm feeling that the current patterns are not able to sustain the addition of more features without causing serious deterioration of the overall quality of the code. On the flip-side, I also feel a huge sense of relief when I refactor something that was starting to slip into disorder and then manage to create the right abstraction that lets us reign it in. Doing this can be downright euphoric. But is there anything we can pinpoint about the source of the "slippage" to begin with? I think so. I've seen this in so many apps now, that I feel confident that it's a common source of many of these issues. For context, I've built a wide range of what might be considered "ambitious" web apps. Real-time asset geo-location tracking systems, chat apps, video calling apps, the Starbucks PWA, apps that enable you to make and receive real phone calls in a browser, surgery tracking apps, and recently I've been working on an e-commerce PWA for one of the world's largest retailers. These observations seem to hold true regardless of app or team size. Early development tends to go something like this: Initial requirements are identified and a small team builds a tidy, small, simple prototype with no real tech debt based on their understanding of initial requirements. Stakeholders and early users start using the prototype and quickly realize there were several scenarios that were not accounted for. In addition, they also start to generate a whole bunch of "wouldn't it be awesome if we could" feature suggestions. Code tidiness (and often performance) goes to hell as developers scramble to connect previously cleanly isolated areas of the app in the attempt to build those "wouldn't it be awesome if we could" features. These type of features frequently involve connecting previous isolated logic because you realize there is value to be gained by doing so (example below). Later, when the app is launched to real users, steps 2 and 3 repeat, over and over. Or more precisely, they repeat until the developers working on it so dread their work that they either walk away, stop caring, or stop being able to actually deliver them in a timely manner. 2-5 years after the initial build someone comes along as says: "this is all garbage, we need to re-write it." And... the whole cycle starts again. To some degree, this is inevitable, and not all bad because sometimes you just need a fresh start. But, perhaps if we did things a little differently we could at minimum extend this cycle a bit. So, if we look at the above sequence there is a precise moment where things seem to go off the rails. It happens at the point where the original assumptions are challenged. Can we perhaps define the point more clearly? I think so. As I hinted at above, more often than not, the precise point where "loss of control" over the app begins is when someone realizes the potential benefit of connecting logic from two previously disparate areas of the application. I've seen this so often that I've started to believe that most of us are drawing the wrong boundaries when we first build an app. I think this happens because we're too focused on what data the app will consume instead of how the data will interact in our app. This is a natural consequence of identifying the data we'll be fetching in our apps. But I think we tend to stop here for a couple of reasons: Our brains have a bias toward broad generalizations and simple solutions. We at least know what data models we're dealing with, so we build around that. We don't fully understand what the real requirements should be until we try to build a solution so we build around what we do know: the data models we can get from the data APIs. When we're first starting imagining a brand new application, it's relatively easily identify the types of data we'll have to keep track of and it often fits into neat little buckets. The trouble is: The whole reason for building a UI in the first place, is often to enable us to manage the interactions between these different data types! Let's make this less abstract Let's say you're building an e-commerce experience. You know you'll have products in various categories. You know you'll have users, and shopping carts, and payment methods etc. You know that people will have to search for products, add them to a cart, and then checkout. So, from a data perspective we'll have: Products Categories Carts Users From a process perspective: People will search They'll add to cart They'll checkout Simple, right?! So, as developers and product managers we have some really clean, simple lines we can draw, right? We'll obviously need an API that returns collections of products as search results, and as items in your cart. So when splitting up the work of building a UI, we kind of let the data types split the UI concerns too. Part of the UI will be all about searching for products. There's one clean boundary. Then we'll need a product detail view that pulls in relevant product data like full descriptions, images, and user reviews. Then we'll need a cart view, and a checkout view. That all sounds good, let's divide an conquer! We can have one team focused on building the API, we'll have a product model with a search API. We'll have a set of categories which we can group products into. From the UI side of things we'll need a search page, an item detail page, a cart page, and a checkout page. Let's have a "shopping cart" team, a search page team, a homepage team, etc. All the pieces are cleanly separated. Because it was logically assumed that a user would cleanly jump from one part of the app to the other. Again, our bias toward simple solutions means we opt to split along easily identifiable major sections of the app. Great! The code is beautiful, teams are operating independently: everybody is feeling good about themselves. Then, a smart UX designer comes along and says: "You know, we really ought to have customers be aware of their cart the whole time. It's like walking around in a store, you have your cart with you and you should be able to metaphorically glance down at it whenever you want without leaving your shopping experience." This is hard to argue with. Of course that's a better experience. So, now what? You already have a completely separate team managing the "cart" experience. They own everything that happens at the yourstore.com/cart page. It's ok, we can hack this. Let's add this bit of logic to the persistent header that shares a bit of code that checks and show the cart count, no big deal, right? Well, it chips away a bit of architectural beauty. Now, we have to share some piece application logic for fetching and retrieving the shopping cart data in a way that can be used by the cart team, but also can be used by every other team. But, whatever, we deal with it. It produces customer value, let's ship it! The UX designer is happy, the new feature is shipped and everybody feels good about themselves again. But soon, when people get a taste for this ability they say "well wouldn't it be awesome if we used our knowledge of what was in the cart to enhance and customize, well... pretty much the whole experience!" It goes on and on: "You know, we really ought to let users not just see how many items are in the cart, but also actually pop it open to see items from anywhere." "Well, if we know what's in the cart, maybe we can use the home page to show related products to what you've already added." "Oooh, what about if they re-visit the item detail page of an item that's already in their cart, perhaps we should remind them that they've already added it." "Well, if we're showing suggested items related to what's in the cart, we should have it update those suggestions each time they add something and then include those suggestions throughout their shopping experience." "Oh, but what if they add something to their cart directly from the area of the app that is showing that list of suggested products? In that case, we don't want to refresh that list right away... just imagine if they're in the middle of adding two things and then it disappears from underneath their hovering fingertip. So, can we make sure we only refresh that list once it's out of view?" Can't you just feel your internal barometer starting to build pressure? Our previous lines of separation break down. The idea of the item detail page showing a single product is no longer a standalone concept. Now we really care about that product in the context of the cart and the context of other things we may want to search for, etc. This may seem annoying at first, but should not come as a surprise. Often times the entire purpose of an app / UI is to handle the intersection of the various types of data. A few examples off the top of my head: An e-commerce app is about combining products in different categories into shopping carts. Then connecting those shopping carts with customers with certain payment methods, order histories, and preferences. A "contacts" app is about organizing people you know into favorites, groups, and your interaction history with them. An anesthesia charting application is about combining a patient record with medications administered, vitals recorded, and people involved. I would argue that UIs, by their very nature, are all about connecting disparate types of data! Yet somehow, when building UIs our data structures often fail to account for this. So what happens? Progress slows as engineers start to push back on what really would have been fairly simple change requests if it had been built differently. What could we have done? We can't anticipate what specific features we're going to be requested to add. But, perhaps we can avoid drawing such distinct lines to begin with? Perhaps we can increase our flexibility somehow? I'd like to make another observation here: Figuring out how to render stuff is not the hard part of building a rich, interactive UI! Regardless, of whether you use Vue, React, Preact, or whatever. Let's be clear, the difficulty is not the how-to-generate-DOM part. The hard part is managing the intersection of interrelated but seemingly distinct state in a way that doesn't cause what I like to call "state bugs". Where something ends up in an odd state. For example, one part of the UI reflects state that is no longer true. What if our mental models are wrong? The three predominant approaches we've seen to building stateful UIs on the Web in the last decade have essentially been: Model and Collection objects (think Backbone.js, Ember.js, etc.) Component all the things! (React state, very little code lives outside the files that defining components) State "god objects" paired with something that is a "view layer" this could be anything, but commonly Redux and React. Option #1 is the embodiment of the first part of the shopping cart example I mentioned above. Apps that I've personally built this way (which is quite a few, I first wrote about adopting backbone in 2010) always got messy when trying to share models between collections, or somehow associate models with one another. The one really strong point in favor of this model/collection approach is it often perfectly mimics the data structures we get from data APIs. Option #2 works great for simpler stuff. In my option, one of the main reasons the React programming model is so successful is it solved the nested component problem. Before React we were always hacking together "child views" but we always had to manage them ourselves. But the shortcoming with using components for everything is that it doesn't directly address the problem of state that clearly is not directly tied to a single component. With React 16+ we have the new context API. This helps pass state around, but still doesn't directly help us solve the structuring and management of "global app state." From what I've seen, attempting to do everything in components frequently leads to a giant interconnected mess of components that are hard to to reason about as a whole. Option #3 has enabled some incredibly cool demos like time traveling, state rehydration, etc. But it has been problematic for many to implement in the real world. As it turns out, dealing with a single app-global state object comes with its own set of challenges. Gaps in functionality of Redux (and similar state tools) have been filled by third party libs and many times, the core idea of having a "god object" is a bit at odds with other popular libraries. For example, redux and React-Router have not always played nicely together. And other perceived gaps in functionality have led to a large crop of supplemental libraries such as redux-saga, redux-loop, and Immutable.js. Also, the process of connecting redux to something like react has historically been a bit messy. After after writing a ton of mapDispatchToProps, mapStateToProps suddenly the idea feels a little less ideal. Frankly, that summary of the three options is a bit disappointing. Do all our models just suck? I don't think so. Perhaps we can combine the best ideas from all of them? Use the mental framework of "collections" to fetch data from APIs. We don't have much choice here, this is just kind of how data APIs have to be structured. But let's skip the whole "turn each item into a model object" part. Use a component hierarchy like React, or my favorite: Preact. But, let's use it for what it's best at, which is rendering a tree of visual components. Let's use local component state for things that are truly local to the component and its children. Usually local state is perfect for things like transient UI state, such as whether or not a particular line item in a list is temporarily in "editing mode." Or for tracking form values and form field error states while the user is in the process of filling out a form. Use the "god object" approach to aggregate all the various data fetched by APIs as well as any other "app state" that needs to be shared between components. From my experience, plucking the best patterns from each is great. But, there are still shortcomings we have to make up for in the "god object" approach. Luckily, I've found that those things can be worked around. Unlike trying to hold strict boundaries between different data-types or "pages" within an app, there are other boundaries we can draw that are much more defensible because they're more structural instead of being linked to the app's feature set. They are: Don't introduce any other sources of truth. If you're doing the "god object" thing. By all means, don't have multiple "gods." If you need data from something like a browser API write some glue code that mirrors the relevant state from the browser into your "god object." This includes things like the current URL (you can read more about how I do routing with redux here). Always read state through an abstraction. There's a concept called "selectors" that is essentially a function that takes the state from the "god object" and returns just the slice of state you care about for a given thing. By always reading state through selectors you can retain flexibility in how you choose to structure the "raw" data of the "god object". Also, selectors enable you to "subscribe" to a very specific subset of changes within your "god object" enabling efficient rendering. The most common example of this pattern is probably: reselect. Precisely never store derived state in the "god object." Only store state in its rawest, purest form. You can decorate and enrich it as it's being read through selectors. This will save you a lot of headaches. For extra credit: never use componentDidMount, or something similar to initiate data fetching. Because, contrary to popular opinion, whether or not a particular component is mounted has very little to do with whether or not the app actually should consider fetching the data. The shopping cart example above is a good example of this. Regardless of whether the user is looking at their cart or not, we want to know what's in their cart because we can enhance the entire rest of the experience based on that data. Instead, ideally we want to be able to just declare a maximum stale age that we're ok with, and it should refresh the data behind the scenes whenever it's too old. To accomplish this we can use selectors to subscribe to our "god object" and look for conditions which should trigger other changes. If you're able to follow those rules, just think about what you now have: You have a single source of truth containing everything your entire application needs to be aware of. You can easily combine state from completely disparate sources in whatever fashion you need, as you read it through selectors. You can write code that that responds dynamically to any possible condition that may need to be responded to. You can export and recreate this entire "god object" at will to recreate any state your app can be in. You can init and "run" an entire functional application without any visual components having been built whatsoever, this is analogous to running a browser in headless mode. You can build an entire data layer for an application before design has even decided how anything is going to look. To be clear, these are not hypotheticals these are the approaches I've used to build the last five apps I've been involved in. Applying these ideas to our e-commerce example note: for brevity I'm glossing over some of the implementation details. I've written a whole book that goes deeper if you're interested. Requirement #1: The cart should be fetched and be available no matter where you are in the app. A top-level key in the "god object" tracks the current cart data. When the app boots up, the mere lack of cart data is sufficient to trigger a fetch of cart items. Once fetched, it is stored along with a timestamp indicating when it was last successfully fetched. Now, let's say we want to show a cart count number in the header. We can now take that raw cart items data (which is an array of objects) and write a selector to derive a cart number to show like this: // select relevant slice of data from our "god object" selectCartItemsRaw: state => state.cartItems.data, // a selector that always returns a usable number // of cart items regardless of the state of cart // items. Even if we just haven't fetched the // cart items yet. selectCartItemCount: createSelector( 'selectCartItemsRaw', items => items ? items.length : 0 ) Ok, so now we have a selector the header nav bar component can subscribe to, in order to always render a count. If we later determine that we want our total number of items to take into account multiple quantities of the same item, we can easily change our selector to now determine that value a little differently. That's literally a one-line change: // now we change the existing selector to // instead sum up the `quantity` attribute // of our cart items. selectCartItemCount: createSelector( 'selectCartItemsRaw', items => items ? items.reduce( (total, item) => total + item.quantity, 0 ) : 0 ) Requirement #2: we should fetch a list of suggested products every time the cart contents changes. If you were building it, how would you do this? If you're like most developers I've seen, you would create a "fetch related products" function that you manually called after each "add to cart" operation, right? But, that's super procedural, what if they added one more item of the same thing that the cart already had so the "list of items" didn't actually change. What if they remove something? Now you have to manually run it after that too, right? Soon, we're daisy chaining things and directly coupling one type of fetch with another. Maybe we can do this a little differently. First, maybe we just need to extract a list of product IDs in the cart, since that's what's sent to the recommendation API. No big deal, we add another selector: selectCartItemIds: createSelector( 'selectCartItemsRaw', items => { if (!items) { return null } return items.map(item => item.id) } ) Now, let's write something that triggers the fetch if we have cart item ids that we haven't already fetched, and we're not already in the process of fetching them. Again, I'm glossing over some of the details, in order to focus on the ideas. But essentially this is a selector too, but it will just lead to trigger an action if certain conditions are met. reactShouldFetchRelatedProducts: createSelector( 'selectCartItemIDs', 'selectRecommendedItemsRaw', (cartItems, recommendedItems) => { if (!cartItems) { return } const idString = cartItemIds.join(',') const idsHaveChanged = recommendedItems.lastFetchIdsString !== idString if (idsHaveChanged && !recommendedItems.loading) { // if this doesn't make sense to you, don't worry // the point is we're triggering another action // as a result of a set of conditions return { actionCreator: 'doFetchRecommendedItems', args: [idString]} } } ) Now, anytime an item with a new item ID is added or removed from the cart it will update the recommended products list, sweet! Ok, so now what? Oh, right, we said we don't ever want to update the list of recommended products while the user is looking at their cart, which is where those recommendations are shown. Fortunately, our "god object" also knows the current URL pathname and we have a selector that returns just the pathname part of the URL. So all we have to do is add one more condition: reactShouldFetchRelatedProducts: createSelector( 'selectCartItemIDs', 'selectRecommendedItemsRaw', 'selectPathname', // <- This is new (cartItems, recommendedItems, pathname) => { // we now also check to make sure user is not looking at their cart if (!cartItems && pathname !== '/cart') { return } const idString = cartItemIds.join(',') const idsHaveChanged = recommendedItems.lastFetchIdsString !== idString if (idsHaveChanged && !recommendedItems.loading) { // if this doesn't make sense to you, don't worry // the point is we're triggering another action // as a result of a set of conditions return { actionCreator: 'doFetchRecommendedItems', args: [idString]} } } ) Now, our recommendations will magically fetch whenever cart item IDs change, but only if the cart page is not currently open. Requirement #3: modify the item detail page if the item is in the cart. Crap! Now we have another two, seemingly distinct, concerns that all of a sudden care about each other. Again, our "god object" knows what URL they're on, so we can know which item page they're looking at because the item ID is in the URL. So, we can select the "activeItemId" (realistically we have already written this by the time new new requirement comes along, but it probably looks something like this: selectActiveItemId: createSelector( 'selectPathname', // url pathname 'selectRouteParams', (pathname, routeParams) => { if (!pathname.startWith('/product')) { return null } return routeParams.itemId } ) Ok, so let's assume we've also got a selector called selectActiveItem that actually returns the data we've fetched for that item. Read the code comments in this block carefully: selectActiveItem: createSelector( 'selectItemDataById', // grabs all item data we have 'selectActiveItemId', // the one from above 'selectCartItems', // the current items in cart (itemDataById, activeItemId, cartItems) => { const foundItem = itemDataById[activeItemId] if (!foundItem) { return null } // now if we want to mix in what we know about the cart // we can do that without manipulating the "raw" state at all // let's see if we have one const cartItem = cartItems.find(item => item.id === activeItemId) // now we can decorate our active item with the corresponding cart item // if it exists. return Object.assign({}, foundItem, {relatedCartItem: cartItem || null}) } ) Here you can see how we've successfully combined two completely different types of data. Or to put it into MVC / Backbone.js terms, two models from two different collections are now merged without truly merging their sources. Now, the page that renders the item detail page just has to check for the existence of a .cartItem property to render something different in that scenario. You can start to see how mashing together seemingly disparate parts of state can now be done very arbitrarily. This is a huge strength of this approach! Now if some product manager decides that on the third tuesday of the month, and a user have more than 50 items in their cart, and if three or more of those items qualify for a warranty, then we need to render a congratulatory message. All of a sudden, with this type of approach, it doesn't really sound all that scary to implement. We just define a selector that answers each one of those questions. Then use those as inputs to another selector that checks if all those are met. We haven't even had to mess around with manually firing data fetches. We've defined the conditions that should trigger a fetch, and we've seamlessly stitched it into the data that is already being used to render the item detail page. This also, very importantly, means that our view components, (React, Preact, View, Web-Components, etc.) can actually be quite simple. They just have to focus on rendering the right stuff, given the properties they're given. Conclusion Hopefully I've at lest helped illustrate at a high level, the benefits and flexibility that we can get by changing the boundaries we use when building UIs with complex and ever-evolving feature sets. You may be thinking this all sounds nice, Henrik, but... does it scale?!?! In my experience on 5 different apps, Yes. Can I do SSR with these ideas? Yes, I'm doing this for a big client right now. Is it performant? That's a bit of an tangential question, because it's also possible to do really stupid things if you misuse these patterns, but yes, in my apps this has not prevented me from consistently building apps with lighthouse performance scores in the high 90s. There are obviously still implementation details I'm completely ignoring here. To fully explain all those things and the tools and libraries I actually use to do all of this in practice I'd have to write a whole book, which I actually did and made available online for free: read.reduxbook.com. Hope I've at least given you something to think about. I'm just trying to share the ideas that are working well for me. I figure if we all keep doing that, everybody wins by learning from each other's experiences. Let me know what you think. Easiest way to reach me is on Twitter: @HenrikJoreteg. Thanks for reading!

Building Honey Badger Web Apps

This is a chapter from my new book Human Redux which is now available! Chapter 11: Reliable apps As a user, nothing will more quickly sour me on a piece of software than flakiness. When we try to get someone to use our software, to some degree, we're asking them to trust us. If they're going to enter their data, spend their precious time, or hope to get some value out of the app we've built, they have to trust that it's going to do what it's supposed to do. How is trust built? Through experience, right? Consistent experiences over time lead to trust. Historically, web developers haven't really had to care about reliability quite to the same extent as folks building native applications, since the web itself has always been a little untrustworthy. Our Internet connections are far from perfect, browsers have different quirks, and at the end of the day, our users can always just hit "refresh" if we end up in a broken state. To this day, when I've written anything of significant length into a <textarea> I'll instinctively copy it to the clipboard before hitting "submit" because I don't trust that the developers will have taken care to make sure I don't lose my work if it fails to submit! Sure, if you're building simple websites that level of inherent flakiness might be acceptable because users expect it. But, at the point where you're asking a user to use your app to accomplish something that matters to them all of a sudden this becomes unacceptable. The Web has a trust problem When you type a URL into a browser and hit "enter" what level of confidence do you have that it will quickly load and get you what you're looking for? By contrast, when you open Instagram on your phone what level of confidence do you have that it will open quickly and will at least show you what you saw last time you opened it even if you don't have an Internet connection? Quite a different level of trust, right? It used to be that this difference could be attributed entirely to the differences between the platforms. On the one hand, The Web won't typically get you anything if your connection is failing, on the other hand, Instagram, the app, is already downloaded and installed locally and the data it has fetched is also stored locally. So it shouldn't be too surprising that Instagram can achieve a higher level of reliability, right? Here's the thing: That excuse is gone! Today's web platform allows us to build apps that can: Be installed on the device Be opened and run from cache first, before even trying the network Fetch and store data locally The inherent reliability advantages of native apps are gone. But as a group, web developers haven't necessarily embraced that mindset. But, if we want to build trust with our users, it is downright irresponsible for us to assume we'll always have a fast, reliable connection. We simply cannot assume that we'll always be able to fetch what we need on demand. Let me say that again: If we want to build trust with our users, it is downright irresponsible for us to assume we'll always have a fast, reliable connection. You may have heard the phrase "Offline first" used to describe these types of approaches. But "Offline first" is somewhat confusing since many people read that think "Oh, I don't need offline support." But, what I'm discussing here is reliability. Everyone wants reliability. Consistent, reliable experiences build trust, and people spend their time (and money) with companies and services they trust. Rebuilding trust If I tap an icon on my phone, I expect that app to open. Period. Sure, it may not be able to connect to fetch what it needs but it damn well better open and at least tell me that it couldn't get what it needed. Users have that expectation of things that are installed. In fact, that's arguably what the word "installed" actually means to a user. Sure, a modern web app that you open in a browser may have been built to be capable of working offline, and truly be run locally, as if it were installed. But, we're never in a million years going to teach users to open their browser, type in a URL and hit "Enter" when they know they don't have an Internet connection! It's simply not going to happen because in that scenario our app isn't "the app" they're running; the browser is the app they're running. As a result, if we even want a shot at creating the same level of trust, we need our apps to exist outside of the browser. Users have to be able to install our web apps on their devices. Fortunately, that's now possible with Progressive Web Apps. I won't get into the nitty-gritty of building PWAs here, but if you're not familiar with them, I'd urge you to read the "Betting on the Web" blog post that I've included in the Appendix. I would also argue, however, that at the point where we've recognized that we're building real apps, that realization should affect how we approach data fetching. Trustworthy data fetching If you've built web apps, you've no doubt fetched data from a server from JavaScript that's running on a web page. Back in the day, you may have done it like this: $.ajax('https://example.com/some.json', { success: function(data) { console.log(data) } }) If you were diligent, you'd also handle the error case. More often than not "handling the error" involves popping up a message to the user saying something like "Sorry, there was an error fetching your data." I suppose this is marginally better than doing nothing, but sheesh, in most cases you've essentially left your user at a dead-end street without a map; not exactly great for building trust. Not to mention, in a mobile world, a failed request should not be a surprise! It's not an edge case. It's not even strictly an exception because frankly, it's not all that exceptional. As we've discussed in previous chapters, if we want to build reliable applications we have to decouple the data from the views that show them. If we've coupled things so that displaying a component always causes a fetch that breaks the cache-first behavior. There are cases where that's sufficient, but often it sells users short. What's the problem with just fetching data when a view loads that needs the data? Imagine the following example: class MyPage extends Component { componentDidMount() { this.setState({ loading: true }) fetch('https://example.com/some.json') .then(res => res.json()) .then(data => { this.setState({ loading: false, data: data }) }) } render() { if (this.state.loading) { return <p>Loading...</p> } return <div>{data.map(item => <p>{item.name}</p>)}</div> } } In this case, anytime this component is added to the page it's going to immediately fetch data while showing a "loading..." message. The trouble is, if the component was added, removed, then quickly added back again it will still show loading... even though we just successfully fetched the data 2 seconds ago! And sure, you can use various simple loader helpers that will mitigate the need to re-fetch. But they're usually quite simplistic and doesn't address the issue of coupling view and behavior. As we've said, these simple approaches may have been OK for the old web, but if we're building real apps, we can do better. A rather ambitious, yet real-life example While I was prototyping the PWA for Starbucks before it became a "real" project, I ran into an interesting feature in the mockup design. The goal was to show what song was currently playing in a given Starbucks store, that is if we thought you were in one. Conceptually the various APIs required to pull this off were all available. But, the process was not exactly straightforward; quite the opposite, in fact. Here's what it had to do: Call an API to return "cards" to be shown on the landing page of the app. If we got a "now-playing" card back with sufficiently high priority, we'd attempt to fetch the song that was now playing. First, we'd need to determine if we had permission to ask for geolocation. If we didn't, we wanted to display a "card" in the UI casually explaining the cool feature we could provide if they would give us permission with a button they could use to grant us permission. If we had permission, then we would trigger a geolocation request of the user behind the scenes and hope it was successful and sufficiently accurate (which is anything but guaranteed with this API). If we had geolocation that was within our accuracy threshold and not too stale we could use that to trigger a request to the API that would give us the closest store to that geolocation. If there was a store nearby and within distance threshold, we could reasonably assume the user was probably in or at least near the store. Now that we have a store ID we could attempt to call the API that told us what music was playing and use the result to show the right image and song info. That's quite the cascade of things that need to occur, right? Plus, any number of them were known to be likely to fail. Geolocation permission could be denied. Geolocation could fail. Geolocation could just not be accurate enough to trust. There may not be any store nearby at all. There may be a store returned but is often too far away to assume the user was inside. We may have correctly guessed the user is in the store, but that particular store may not have "now playing" information available (not all stores did). Also, often, when a customer enters a store, their phone will start to try to connect to the wifi. But, they may not have accepted terms of use. So if a customer is walking into a store, the network may be working outside, but stop working as they near wifi range. So, by no fault of Starbucks', any number of things could go wrong here. How on earth do you even go about building something to handle this reliably? How we tackled it First, as you probably guessed if you read the previous chapter, all the logic was contained in selector functions. Selectors could be used to inspect what point in the process we were. Unless all the prerequisite conditions were in place, the next actions would not be dispatched. This alone, was incredibly effective at just ensuring only the next possible thing would be triggered. Also, at each point in the process where any step of the process was successful the result would be persisted locally by means we'll discuss in the next chapter. Then, on startup, the application's initial data that we passed to createStore would always contain the contents of the reducers that had been successfully persisted. This meant we could march forward in this chain of events without ever needing to take a step back. It also meant that even if a user refreshed the browser or got disconnected because their phone was trying to jump to WiFi, the process would always be able to pick up where it left off, rather than having to start from the beginning. Recovering from errors Since, as we stated, things were likely to go wrong I didn't want the whole thing to fail because part of it failed. Instead, I wanted it to wait a little bit then keep trying. In order to support this in addition to storing the result of the data fetches, I also kept track of the metadata about the request. To be more specific, the reducers stored state that was structured like this: const state = { // the actual data, the payload of the // successful requests data: null, // a flag to check whether currently loading loading: false, // timestamp of last successful fetch lastFetch: null, // timestamp of last error lastError: null, // The type of error that occurred error: null } The reducers would then do this type of thing: const reducer = (state, action) => { if (action.type === 'SUCCESS') { return Object.assign({}, data, { data: action.payload, // CAVEAT: Using Date.now() here // makes the reducer impure. Personally, // I don't mind, but it could certainly be // argued that this value should be passed // in as part of the action instead. lastFetch: Date.now(), loading: false, // clear any previous error data lastError: null, error: null }) } // other conditions return state } In this way, by tracking this metadata along with the data itself, we had enough information in selectors to determine what the next step should be. If sufficient time had passed since the last error, the data was too stale, and we weren't currently fetching, we could in a reactor compare those timestamps to the "app time" and dispatch an action creator that give it another try. With all these pieces combined I could build a very robust solution that would: Retry if there were network errors. Give up if there were "permanent" failures such as the user blocking geolocation. Persist any time it succeeded and automatically pick up where it left off. Automatically determine what to do next based on how long it had been. Essentially, it was a "honey badger." You could just let it start, then sit there and refresh the browser but it would still keep marching forward until it completed. I think this type of resilience should be the standard for PWAs. At first it a problem like this sounds quite daunting, but by breaking down the problem into smaller pieces, we can tackle them one by one and ultimately it's quite doable. Higher order reducers You may have noticed in the example reducer where I just handled one action.type things get a little busy. Imagine if you have 4 or 5 different resources where you have to do this bookkeeping along with each reducer that needs these kinds of capabilities. Humans are inconsistent. Handling each of those conditions and tracking all that metadata correctly in many different reducers in our app sounds like a recipe for lots of evil, subtle, hard-to-track-down bugs. We're programmers, no? Let's solve this with some programming. Instead of handwriting these complex reducer functions, perhaps we can write a function that will generate a reducer for us? You could call this approach "higher order reducers." To do this, we need to write a function that will take a "base" action type and then return a reducer that handles the state changes for the error and success cases. Let's target an API like this: import getMetadataReducer from 'get-metadata-reducer' import { createStore, combineReducers } from 'redux' const store = createStore( combineReducers({ usersReducer: getMetadataReducer({ baseType: 'FETCH_USERS' }) // ... // ... other reducers // ... }) ) By using a convention for how our actions are named, we can write a function that supports that target API: get-metadata-reducer.js export default ({ baseType }) => { const START = `${baseType}_START` const SUCCESS = `${baseType}_SUCCESS` const ERROR = `${baseType}_ERROR` const initialData = { data: null, lastError: null, error: null, lastFetch: null, loading: false } // here we're returning our customized reducer return (state, action) => { if (action.type === START) { return Object.assign({}, state, { loading: true }) } if (action.type === SUCCESS) { // if successful we store our data // store the lastFetch timestamp // clear out any errors // and set loading to false return Object.assign({}, state, { data: action.payload, lastFetch: Date.now(), error: null, lastError: null, loading: false }) } if (action.type === ERROR) { // we still want to leave existing // data intact as well as "last fetch" // which would let us determine if the // data is stale or not return Object.assign({}, state, { lastError: Date.now(), error: action.error, loading: false }) } return state } } In this way, we can take something that sounds tedious, complex, and error-prone and turn it into something we can isolate, write unit tests for and know works consistently. Similarly, we can create helpers for our selectors that can determine if we should fetch something based on this now consistent data structure and the current app time. The "honey-badger" example available at: reduxbook.com/honey-badger is built on these ideas but uses tools from redux-bundler to do it. The resilience of this approach enables us to build some of those "riskier" features in a way that is ultimately reliable. Mixing higher-order reducers into your own reducer One problem with generating reducers it that reducers are not always so generic. You may well end up in a scenario where you have a reducer where you want to do this type of metadata tracking, but you also need to handle additional action types. As it turns out, we can manually compose the generated reducer with our own to accomplish this. If you recall from Chapter 3 on updating state, we can use combineReducers to nest reducers however we wish. Using combineReducers to keep our generated reducer separate from our customizations is probably cleaner, but that's not our only option. Let's manually use our generated reducer inside a custom one to extend it with the ability to handle other action types as well: // just using the file we defined above import getMetadataReducer from './get-metadata-reducer' import { createStore, combineReducers } from 'redux' const generatedReducer = getMetadataReducer({ baseType: 'FETCH_USERS' }) // we will write our own reducer here const userReducer = (state, action) => { // we can just pass our state and action // through the generated reducer first. state = generatedReducer(state, action) if (action.type === 'SOME_OTHER_ACTION') { return Object.assign({}, state, { otherThing: true }) } return state } const store = createStore( combineReducers({ usersReducer // ... // ... other reducers // ... }) ) As you can see, since we're just dealing with simple functions here we can combine them however we wish. Note: If you needed to add something to the initialState inside the generated reducer that can also be done. We would change our getMetadataReducer helper to return an object containing both a generated initialState object and the reducer. Then, when we used it, we'd grab the resulting initialState along with the reducer. We can modify that and use it as part of our reducer too as shown below: // again grabbing the file from above, that now has // has been modified to also return `initialState`. import getMetadataReducer from 'get-metadata-reducer' import { createStore, combineReducers } from 'redux' // now this would be an object with both initial // and reducer, so we could grab them as variables const { initialState, reducer } = getMetadataReducer({ baseType: 'FETCH_USERS' }) // here we can modify the initial state returned from // our helper. const initialUserState = Object.assign({}, initialState, { otherThing: false }) // Now we'd pass our updated initial state // as the initial state here: const userReducer = (state = initialUserState, action) => { // remember `reducer` here is the one // generated by our helper above. state = reducer(state, action) // Here we can handle additional action types if (action.type === 'SOME_OTHER_ACTION') { return Object.assign({}, state, { otherThing: true }) } return state } Chapter recap Reliability is not the default on the web, this needs to change with PWAs, or we will continue to have a trust problem on The Web. Coupling data fetching to components doesn't allow for the reliability we're aiming for. I provided a real-world example of trying to show the "now playing" card in the Starbucks PWA powered by very seemingly complicated waterfall of things that had to occur reliably. I showed how to store metadata along with your data in reducers to enable your app to recover from errors. Higher-order reducers can be used to tackle tedious "bookkeeping" problems while significantly reducing the surface area for bugs. We can compose the generated reducer function into another reducer to customize behavior to support handling other action types as well. For an example of what you can do with this approach see: reduxbook.com/honey-badger If you enjoyed this the whole book is now available. It's called Human Redux buy it at https://reduxbook.com!

A PWA that Spawns PWAs, Oh My!

TLDR; The ability to build apps with web technology isn't just another way to build the same thing. It enables a whole new category of services that didn't make financial sense previously. As proof, I also introduce a real service that I've built for non-profits called Speedy that generates PWAs on the fly and is built on these ideas. If you've seen any of my writing about PWAs (Progressive Web Apps), you already know that I'm a fan. That is because I believe that the core idea of building ad-hoc, app-like experiences that don't require an install step is transformative to how we use our Fancy Pocket Computer Thingies™️. If you want to get a background on PWAs and why I think they're such a big deal read or watch my "Betting on the Web" post/talk. But, the short version is that The Web has finally gotten "good enough" for making first-class app experiences. A few quick supporting anecdotes I've been happily using the Twitter Lite as my only Twitter client on my phone since it was first launched. I recently I did the same for Instagram, their PWA is coming along nicely and even though it may lack a few features for the time being, I've made the switch. I also recently helped Starbucks build/ship their PWA (U.S.-only for the time being). There are many, many more, examples of companies investing in PWAs. All this to say this isn't theoretical; it's already happening. That's nice Henrik, but users don't care how the app was built! You're absolutely right. Once it's on their device, they don't care whether it's powered by web tech or is running native code for their platform. But, that's actually kind of the point! We can now build the same experience and deliver it in less time than it takes to load the typical boring ol' website. I would argue that users do care about whether or not they have to go install stuff from an app store or not, but I digress. For the purposes of this post let's assume users don't care at all. What this does, however, is change the types of things that make sense for us to build as developers. Let's take an example of a local bike shop. Should they have an app? If we're talking about an app in the traditional sense: absolutely not. There's a very low chance that a bike shop owner could justify dropping $30k+ to a software company to build a custom native app for her. I mean, why would she?! Realistically, unless there's some serious value or incentive she can offer to customers for installing it, they're simply not going to do it. It is extremely unlikely that it investing in custom app development would be a wise investment for her. But that's not to say that there aren't aspects of her business where she could benefit from providing a greater level of interactivity with her customers. What if instead of her bike shop website just being a simple page with contact details and a pic of the shop it was… wait for it… ✨a PWA!✨ It could load just as fast as a simple site, but also let her do more. It could show off current inventory, highlight sale items, and maybe let loyal customers track their purchases over time to earn rewards? Or let folks track status of their bike repair. Perhaps there are a few of her most loyal customers who would actually want to add her shop's PWA to their home screens? Or maybe she could have a way to do push notifications about special events, or group rides. For many small businesses, keeping loyal customers engaged and coming back is their life-blood. Therefore, anything that helps them do that is hugely valuable. There are so many use-cases that don't justify the cost of custom software development but could still benefit from a an app-like experience! So what are we supposed to do with this information? Perhaps we can use the inherently dynamic nature of the web to make PWAs for things that previously didn't justify the investment? What I mean is that "bike shop software" can likely be reduced into a fairly constrained and generic set of potential requirements. If she could pay ~$100/mo for a service made for bike shops where she could customize, configure, and publish a PWA that helped her better engage her customers that'd be a no-brainer! There ya go, free business idea: build the ultimate bike shop management tool and make it so each bike shop customer can generate/configure a PWA for their customers. A few more businesses off the top of my head that could benefit from something like this: Hair salons Appointment bookings Push notification reminders Loyalty programs Auto repair shops Tracking status of repair Authorizing proposed fixes Lawyer's offices Scheduling appointments Delivering consultations via WebRTC video Cities Payment of public utilities Public parking, fees and fines Doctor clinics Appointment scheduling Requesting a call back for a medical question Viewing medical records Viewing/paying medical bills Churches Member directories Online giving Publishing events Schools Updates/notifications Parent teacher communication and scheduling Coffee shops Order-ahead functionality (a. la. Starbucks, but for any coffee shop) Location-based info, what music is playing, etc. In-store ordering via tablets on tables If you build one of these, and make a lot of money I'd gladly accept 1% of your profits 🤓 . This isn't actually new, Henrik I suppose you're right, you could argue it's just a content management system with some industry-specific functionality baked in. But, doing this with PWAs takes it to a different level because it actually makes it relevant to our mobile lives. There's an entire class of problems that could arguably be solved once for a given industry and PWA tech ensures they can integrate into our mobile experiences. This basic idea has intrigued me for a very long time. That is, the concept of using a web app to essentially generate customized apps for everyone in a certain niche. I first built something like this 8 years ago using jQTouch and Django. Site note: Some of you will note that there are people doing this type of thing with native apps, they're sometimes referred to as "templated apps". I would argue, however, that most of these really shouldn't be native apps at all because without the automatic delivery and discovery via the company's website it's not likely that many people will download, install, and actually use these. Ok that's a nice theory, but is it doable? Well, umm… so I did this. I've quietly built a business that does exactly what I've been describing. Don't worry, unless you're involved with a non-profit you're not the target audience. The service has been in private beta for some time, but is now open to public as of last week. It's called Speedy if you're interested you can read more about what it provides here: https://speedy.gift. But, for our purposes in this post, it lets non-profits configure their own PWAs for giving them money online using Apple Pay, Google Pay, or credit cards. The whole thing is optimized for removing barriers to giving with a smartphone. If you're a U.S. based non-profit organization I'm not aware of any faster or easier way to start accepting online donations. It's simply a dynamically generated PWA served on a sub-domain of their choosing (unfortunately, for the time being it's limited to the U.S. for legal reasons). The very first bit of setup for a new organization in Speedy is to pick a sub-domain. From the instant they complete that step they are now the proud owners of a PWA for their organization. They can literally start to collect donations through their own PWA in less than 5 minutes of setup. This would not even be possible with templated native apps. Also, just to make things interesting, everything in the generated app and the admin itself, for that matter, updates in real-time. This means as you use the admin tool to configure your PWA you can have another device open to your donation app and watch changes being applied to your app as you change them in the admin tool. As anyone donates, all the charts, graphs, and lists in both the donation PWA and the admin site update in real-time. So you configure a PWA with an admin tool? Yep, and obviously, it's a PWA too! I mean, why wouldn't it be?! We use an iframe to embed the PWA you're configuring into the admin tool so you can see your changes applied in real-time. There's no need for us to build a preview of the app, instead we just show you the PWA itself inside the other PWA! So it's not just a PWA spawning PWAs… It's PWA inception! Here's another screenshot showing the admin tool where you can configure the icon and app name: If you're familiar with PWAs you know that these details are configured using a Web App Manifest which is just a JSON file. So, that part is pretty straightforward to generate on the fly. This is all just silly, Henrik! Maybe. But, is it though?! So far, in a small private beta we've helped a handful or organizations collect over $60,000 in donations through those generated PWAs and the feedback has been overwhelmingly positive from both the orgs and the donors. As it turns out, the combination of making it incredibly easy to donate and being able to see progress toward the fundraising goals animate in real-time as donations come in is highly motivating for donors. In addition we also have a view of the fundraising project designed for display on large screens. So, it can put it up on a projector as a live dashboard in front of an audience: Or a TV in a public space: This view is also just a page on the PWA that includes support for going into full-screen mode. This means it's super simple to put it on a projector or just use an inexpensive Chromecast to show it on a TV. The web is so versatile! Same app can be used on the phone or on a huge projector. Pretty sweet, right?! Anyway, If you're involved with a non-profit you should probably go sign up for Speedy to try it out. But, remember how I said this whole approach enables new business models? Speedy is a case in point. If we were generating native app builds, there's no possible way we could let organizations start with zero up-front costs like we do. How does it all work? Really well, thanks! 😜 If you're a dev you're probably wondering how it's built. I'll do more technical breakdowns in other posts, but at a high level it's built with: Preact Redux Redux Bundler (a soon-to-be open sourced thing I made for minimizing redux boilerplate) Firebase Stripe Connect, Stripe Elements GraphQL API for faster-than-Firebase initial fetches The Web App Manifest is served dynamically by the API sw-toolbox for service worker stuff (will likely update to Workbox soon-ish) What's next? There's still a lot I want to do to build on these ideas and techniques. But at this point I've at least proven the core concept to myself. That's why I figured it was time to share it. I'll be blogging more about some of the technical aspects of this stuff in the future and I'm also in the process of writing a second book. The book is about Redux patterns that I've found really helpful in building Speedy and the work I did for Starbucks. Obviously, if you're involved with a non-profit in any way, I'd love to hear your thoughts on Speedy itself. If you have questions or whatnot feel free to email me: henrik@joreteg.com or follow me on Twitter: @HenrikJoreteg. See ya on the Interwebz ❤️

Betting on the Web

Note: I just spoke at Coldfront 2017 about why I’m such a big proponent of the Web. What follows is essentially that talk as a blog post (I’ll add a link to the video once it is published). Also: the Starbucks PWA mentioned in the talk has shipped! 🎉 I’m not going to tell you what to do. Instead, I’m going to explain why I’ve chosen to bet my whole career on this crazy Web thing. "Betting" sounds a bit haphazard, it’s more calculated than that. It would probably be better described as "investing." Investing what? Our time and attention. Many of us only have maybe 6 or so really productive hours per day when we’re capable of being super focused and doing our absolute best work. So how we chose to invest that very limited time is kind of a big deal. Even though I really enjoy programming I rarely do it aimlessly just for the pure joy of it. Ultimately, I’m investing that productive time expecting to get some kind of return even if it’s just mastering something or solving a difficult problem. "So what, what’s your point?" More than most of us realize we are constantly investing Sure, someone may be paying for our time directly but there’s more to it than just trading hours for money. In the long run, what we chose to invest our professional efforts into has other effects: 1. Building Expertise: We learn as we work and gain valuable experience in the technologies and platform we’re investing in. That expertise impacts our future earning potential and what types of products we’re capable of building. 2. Building Equity: Hopefully we’re generating equity and adding value to whatever product we’re building. 3. Shaping tomorrow’s job market: We’re building tomorrow’s legacy code today™. Today’s new hotness is tomorrow’s maintenance burden. In many cases the people that initially build a product or service are not the ones that ultimately maintain it. This means the technology choices we make when building a new product or service, determine whether or not there will be jobs later that require expertise in that particular platform/technology. So, those tech choices literally shape tomorrow’s job market! 4. Body of knowledge: As developers, we’re pretty good at sharing what we learn. We blog, we "Stack-Overflow", etc. These things all contribute to the corpus of knowledge available about that given platform which adds significant value by making it easier/faster for others to build things using these tools. 5. Open Source: We solve problems and share our work. When lots of developers do this it adds tremendous value to the technologies and platforms these tools are for. The sheer volume of work that we don’t have to do because we can use someone else’s library that already does it is mind-boggling. Millions and millions of hours of development work are available to us for free with a simple npm install. 6. Building apps for users on that platform: Last but not least, without apps there is no platform. By making more software available to end users, we’re contributing significant value to the platforms that run our apps. Looking at that list, the last four items are not about us at all. They represent other significant long-term impacts. We often have a broader impact than we realize We’re not just investing time into a job, we're also shaping the platform, community, and technologies we use. We’re going to come back to this, but hopefully, recognizing that greater impact can help us make better investments. With all investing comes risk We can’t talk about investing without talking about risk. So what are some of the potential risks? Are we building for the right platform? Platform stability is indeed A Thing™. Just ask a Flash developer, Windows Phone developer, or Blackberry developer. Platforms can go away. If we look at those three platforms, what do they have in common? They’re closed platforms. What I mean is there’s a single controlling interest. When you build for them, you’re building for a specific operating system and coding against a particular implementation as opposed to coding against a set of open standards. You could argue, that at least to some degree, Flash died because of its "closed-ness". Regardless, one thing is clear from a risk mitigation perspective: open is better than closed. the Web is incredibly open. It would be quite difficult for any one entity to kill it off. Now, for Windows Phone/Blackberry it failed due to a lack of interested users... or was it lack of interested developers?? Maybe if Ballmer ☝️ has just yelled "developers" one more time we’d all have Windows Phones in our pockets right now 😜. From a risk mitigation perspective, two things are clear with regard to platform stability: Having many users is better than having few users Having more developers building for the platform is better than having few developers There is no bigger more popular open platform than the Web Are we building the right software? Many of us are building apps. Well, we used to build "applications" but that wasn’t nearly cool enough. So now we build "apps" instead 😎. What does "app" mean to a user? This is important because I think it’s changed a bit over the years. To a user, I would suggest it basically means: "a thing I put on my phone." But for our purposes I want to get a bit more specific. I’d propose that an app is really: An "ad hoc" user interface That is local(ish) to the device The term "ad hoc" is Latin and translates to "for this". This actually matches pretty closely with what Apple’s marketing campaigns have been teaching the masses: There’s an app for that – Apple The point is it helps you do something. The emphasis is on action. I happen to think this is largely the difference between a "site" and an "app". A news site for example has articles that are resources in and of themselves. Where a news app is software that runs on the device that helps you consume news articles. Another way to put it would be that site is more like a book, while an app is a tool. Should we be building apps at all?! Remember when chatbots were supposed to take over the world? Or perhaps we’ll all be walking around with augmented reality glasses and that’s how we’ll interact with the world? I’ve heard it said that "the future app is no app" and virtual assistants will take over everything. I’ve had one of these sitting in my living room for a couple of years, but I find it all but useless. It’s just a nice bluetooth speaker that I can yell at to play me music. But I find it very interesting that: Even Alexa has an app! Why? Because there’s no screen! As it turns out these "ad hoc visual interfaces" are extremely efficient. Sure, I can yell out "Alexa, what’s the weather going to be like today" and I’ll hear a reply with high and low and whether it’s cloudy, rainy, or sunny. But in that same amount of time, I can pull my phone out tap the weather app and before Alexa can finish telling me those 3 pieces of data, I can visually scan the entire week’s worth of data, air quality, sunrise/sunset times, etc. It’s just so much more efficient as a mechanism for consuming this type of data. As a result of that natural efficiency, I believe that having a visual interface is going to continue to be useful for all sorts of things for a long time to come. That’s not to say virtual assistants aren’t useful! Google Assistant on my Pixel is quite useful in part because it can show me answers and can tolerate vagueness in a way that an app with a fixed set of buttons never could. But, as is so often the case with new useful tech, rarely does it complete replace everything that came before it, instead, it augments what we already have. If apps are so great why are we so "apped out"? How do we explain that supposed efficiency when there’s data like this? 65% of smartphone users download zero apps per month More than 75% of app downloads open an app once and never come back I think to answer that we have to really look at what isn’t working well. What sucks about apps? Downloading them certainly sucks. No one wants to open an app store, search for the app they’re trying to find, then wait to download the huge file. These days a 50mb app is pretty small. Facebook for iOS 346MB, Twitter iOS 212MB. Updating them sucks. Every night I plug in my phone I download a whole slew of app updates that I, as a user, could not possibly care less about. In addition, many of these apps are things I installed once and will never open again, ever!. I’d love to know the global stats on how much bandwidth has been wasted on app updates for apps that were never opened again. Managing them sucks. Sure, when I first got an iPhone ages ago and could first download apps my home screen was impeccable. Then when we got folders!! Wow... what an amazing development! Now I could finally put all those pesky uninstallable Apple apps in a folder called "💩" and pretend they didn’t exist. But now, my home screen is a bit of a disaster. Sitting there dragging apps around is not my idea of a good time. So eventually things get all cluttered up again. The thing I’ve come to realize, is this: We don’t care how they got there. We only care that they’re there when we need them. For example, I love to go mountain biking and I enjoy tracking my rides with an app called Strava. I get all geared up for my ride, get on my bike and then go, "Oh right, gotta start Strava." So I pull out my phone with my gloves on and go: "Ok Google, open Strava". I could not care less about where that app was or where it came from when I said that. I don’t care if it was already installed, I don’t care if it never existed on my home screen, or if it was generated out of thin air on the spot. Context is everything! If I’m at a parking meter, I want the app for that. If I’m visiting Portland, I want their public transit app. But I certainly do not want it as soon as I’ve left. If I’m at a conference, I might want a conference app to see the schedule, post questions to speakers, or whatnot. But wow, talk about something that quickly becomes worthless as soon as that conference is over! As it turns out the more "ad hoc" these things are, the better! The more disposable and re-inflatable the better! Which also reminds me of something that I feel like we often forget. We always assume people want our shiny apps and we measure things like "engagement" and "time spent in the app" when really, and there certainly are exceptions to this such as apps that are essentially entertainment, but often... People don’t want to use your app. They want to be done using your app. Enter PWAs I’ve been contracting with Starbucks for the past 18 months. They’ve taken on the ambitious project of essentially re-building a lot of their web stuff in Node.js and React. One of the things I’ve helped them with (and pushed hard for) was to build a PWA (Progressive Web App) that could provide similar functionality as their native apps. Coincidentally it was launched today: https://preview.starbucks.com! My team at @Starbucks has been building a PWA, and it's now in beta! Check it out at https://t.co/tEUXM8BLgP if you're an existing customer! pic.twitter.com/b9IJ3HbGpx — David Brunelle (@davidbrunelle) September 7, 2017 This gives is a nice real world example: Starbucks iOS: 146MB Starbucks PWA: ~600KB The point is there’s a tremendous size difference. It’s 0.4% of the size. To put it differently, I could download the PWA 243 times in the same amount of time it would take to download the iOS app. Then, of course on iOS it then also still has to install and boot up! Personally, I’d have loved it if the app ended up even smaller and there are plans to shrink it further. But even still, they’re not even on the same planet in terms of file-size! Market forces are strongly aligned with PWAs here: Few app downloads User acquisition is hard User acquisition is expensive If the goal is to get people to sign up for the rewards program, that type of size difference could very well make the difference of getting someone signed up and using the app experience (via PWA) by the time they reach the front of the line at Starbucks or not. User acquisition is hard enough already, the more time and barriers that can be removed from that process, the better. Quick PWA primer As mentioned, PWA stands for "Progressive Web Apps" or, as I like to call them: "Web Apps" 😄 Personally I’ve been trying to build what a user would define as an "app" with web technology for years. But until PWAs came along, as hard as we tried, you couldn’t quite build a real app with just web tech. Honestly, I kinda hate myself for saying that, but in terms of something that a user would understand as an "app" I’m afraid that statement has probably true until very recently. So what’s a PWA? As one of its primary contributors put it: It’s just a website that took all the right vitamins. – Alex Russell It involves a few specific technologies, namely: Service Worker. Which enable true reliability on the web. What I mean by that is I can build an app that as long as you loaded it while you were online, from then on it will always open, even if you’re not. This puts it on equal footing with other apps. HTTPS. Requires encrypted connections Web App Manifest. A simple JSON file that describes your application. What icons to use is someone adds it to their home screen, what its name is, etc. There are plenty of other resources about PWAs on the web. The point for my purposes is: It is now possible to build PWAs that are indistinguishable from their native counter parts They can be up and running in a fraction of the time whether or not they were already "installed" and unlike "apps" can be saved as an app on the device at the user’s discretion! Essentially they’re really great for creating "ad hoc" experiences that can be "cold started" on a whim nearly as fast as if it were already installed. I’ve said it before and I’ll say it again: PWAs are the biggest thing to happen to the mobile web since the iPhone. – Um... that was me Let’s talk Internet of things I happen to think that PWAs + IoT = ✨ MAGIC ✨. As several smart folks have pointed out. The one-app-per-device approach to smart devices probably isn’t particularly smart. It doesn’t scale well and it completely fails in terms of "ad hoc"-ness. Sure, if I have a Nest thermostat and Phillips Hue lightbulbs, it’s reasonable to have two apps installed. But even that sucks as soon as I want someone else to be able to use control them. If I just let you into my house, trust me... I’m perfectly happy to let you flip a light switch, you’re in my house, after all. But for the vast majority of these things there’s no concept of "nearby apps" and, it’s silly for my guest (or a house-sitter) to download an app they don’t actually want, just so I can let them control my lights. The whole "nearby apps" thing has so many uses: thermostat lights locks garage doors parking meter setting refrigerator temp conference apps Today there are lots of new capabilities being added to the web to enable web apps to interact with physical devices in the real world. Things like WebUSB, WebBluetooth, WebNFC, and efforts like Physical Web. Even for things like Augmented (and Virtual) reality, the idea of the items we want to interact with having URLs makes so much sense and I can’t imagine a better, more flexible use of those URLs than for them to point to a PWA that lets you interact with that device! Forward looking statements... I’ve been talking about all this in terms of investing. If you’ve ever read any company statement that discusses the future you always see this line explaining that things that are about to be discussed contains "forward looking statements" that may or may not ultimately happen. So, here are my forward looking statements. 1. PWA-only startups Given the cost (and challenge) of user-acquisition and the quality of app you can build with PWAs these days, I feel like this is inevitable. If you’re trying to get something off the ground, it just isn’t very efficient to spin up three whole teams to build for iOS, Android, and the Web. 2. PWAs listed in App Stores So, there’s a problem with "web only" which is that for the good part of a decade we’ve been training users to look for apps in the app store for their given platform. So if you’re already a recognized brand, especially if you already have a native app that you’re trying to replace, it simply isn’t smart for you not to exist in the app stores. So, some of this isn’t all that "forward looking" as it turns out Microsoft has already committed to listing PWAs in the Windows Store, more than once! They haven’t even finished implementing Service Worker in Edge yet! But they’re already committing hard to PWAs. In addition to post linked above, one of their lead Developer Relations folks, Aaron Gustafson just wrote an article for A List Apart telling everyone to build PWAs. But if you think about it from their perspective, of course they should do that! As I said earlier they’ve struggled to attract developer to build for their mobile phones. In fact, they’ve at times paid companies to write apps for them simply to make sure apps exist so that users will be able to have apps they want when using a Windows Phone. Remember how I said developer time is a scarce resource and without apps, the platform is worthless? So of course they should add first class support for PWAs. If you build a PWA like a lot of folks are doing then TADA!!! 🎉 You just made a Windows/Windows Phone app! I’m of the opinion that the writing is on the wall for Google to do the same thing. It’s pure speculation, but it certainly seems like they are taking steps that suggest they may be planning on listing PWAs too. Namely that the Chrome folks recently shipped a feature referred to as "WebAPKs" for Chrome stable on Android (yep, everyone). In the past I’ve explained in more detail why I think this is a big deal. But a shorted version would be that before this change, sure you could save a PWA to your home screen... But, in reality it was actually a glorified bookmark. That’s what changes with WebAPKs. Instead, when you add a PWA to your home screen it generates and "side loads" an actual .apk file on the fly. This allows that PWA to enjoy some privileges that were simply impossible until the operating system recognized it as "an app." For example: You can now mute push notifications for a specific PWA without muting it for all of Chrome. The PWA is listed in the "app tray" that shows all installed apps (previously it was just the home screen). You can see power usage, and permissions granted to the PWA just like any other app. The app developer can now update the icon for the app by publishing an update to the app manifest. Before, there was no way to updated the icon once it had been added. And a slew of other similar benefits... If you’ve ever installed an Android app from a source other than the Play Store (or carriers/OEMs store) you know that you have to flip a switch in settings to allow installs from "untrusted sources". So, how then, you might ask, can they generate and install an actual .apk file for a PWA without requiring that you change that setting? As it turns out the answer is quite simple: Use a trusted source! As it turns out WebAPKs are managed through Google Play Services! I’m no rocket scientist, but based on their natural business alignment with the web, their promotion of PWAs, the lengths they’ve gone to to grant PWAs equal status on the operating system as native apps, it only seems natural that they’d eventually list them in the store. Additionally, if Google did start listing PWAs in the Play Store both them and Microsoft would be doing it leaving Apple sticking out like a sore thumb and looking like the laggard. Essentially, app developers would be able to target a massive number of users on a range of platforms with a single well-built PWA. But, just like developers grew to despise IE for not keeping up with the times and forcing them to jump through extra hoops to support it, the same thing would happen here. Apple does not want to be the next IE and I’ve already seen many prominent developers suggesting they already are. Which bring us to another forward-looking statement: 3. PWAs on iOS Just a few weeks ago the Safari folks announced that Service Worker is now officially under development. 4. PWAs everywhere I really think we’ll start seeing them everywhere: Inside VR/AR/MR experiences Inside chat bots (again, pulling up an ad-hoc interface is so much more efficient). Inside Xbox?! As it turns out, if you look at Microsoft’s status page for Edge about Service Worker you see this: I hinted at this already, but I also think PWAs pair very nicely with virtual assistants being able to pull up an PWA on a whim without requiring it to already be installed would add tremendous power to the virtual assistant. Incidentally, this also becomes easier if there’s a known "registered" name of a PWA listed in an app store. Some other fun use cases: Apparently the new digital menu displays in McDonald’s Restaurants (at least in the U.S.) are actually a web app built with Polymer (source). I don’t know if there’s a Service Worker or not, but it would make sense for there to be. Sports score boards!? I’m a independent consultant, and someone approached me about potentially using a set of TVs and web apps to build a score keeping system at an arena. Point is, there are so many cool examples! The web really is the universal platform! For those who think PWAs are just a Google thing First off, I’m pretty sure Microsoft, Opera, Firefox, and Samsung folks would want to punch you for that. It simply isn’t true and increasingly we’re seeing a lot more compatibility efforts between browser vendors. For example: check out the Web Platform Tests which is essentially Continuous Integration for web features that are run against new releases of major browsers. Some folks will recall that when Apple first claimed they implemented IndexedDb in Safari, the version they shipped was essentially unusable because it had major shortcomings and bugs. Now, with the WPTs, you can drill into these features (to quite some detail) and see whether a given browser passes or fails. No more claiming "we shipped!" but not actually shipping. What about feature "x" on platform "y" that we need? It could well be that you have a need that isn’t yet covered by the web platform. In reality, that list is getting shorter and shorter, also... HAVE YOU ASKED?! Despite what it may feel like, browser vendors eagerly want to know what you’re trying to do that you can’t. If there are missing features, be loud, be nice, but from my experience it’s worth making your desires known. Also, it doesn’t take much to wrap a web view and add hooks into the native OS that your JavaScript can call to do things that aren’t quite possible yet. But that also brings me to another point, in terms of investing, as the world’s greatest hockey player said: Skate to where the puck is going, not where it has been. – Wayne Gretzky Based on what I’ve outlined thus far, it could be more risky to building an entire application for a whole other platform that you ultimately may not need than to at least exhaust your options seeing what you can do with the Web first. So to line ’em up in terms of PWA support: Chrome: yup Firefox: yup Opera: yup Samsung Internet (the 3rd largest browser surprise!): yup Microsoft: huge public commitment Safari: at least implementing Service Worker Ask them add your feature! Sure, it may not happen, it may take a long time but at least try. Remember, developers have a lot more influence over platforms than we typically realize. Make. your. voice. heard. Side note about React-Native/Expo These projects are run by awesome people, the tech is incredibly impressive. If you’re Facebook and you’re trying to consolidate your development efforts, for the same basic reasons as why it makes sense for them to create their on VM for running PHP. They have realities to deal with at a scale that most of us will never have to deal with. Personally, I’m not Facebook. As a side note, I find it interesting that building native apps and having as many people do that as possible, plays nicely into their advertising competition with Google. It just so happens that Google is well positioned to capitalize off of people using the Web. Inversely, I’m fairly certain Facebook wouldn’t mind that ad revenue not going Google. Facebook, seemingly would much rather be your web, that be part of the Web. Anyway, all that aside, for me it’s also about investing well. By building a native app you’re volunteering for a 30% app-store tax. Plus, like we covered earlier odds are that no one wants to go download your app. Also, though it seems incredibly unlikely, I feel compelled to point out that in terms of "openness" Apple’s App Store is very clearly anything but that. Apple could decide one day that they really don’t like how it’s possible to essentially circumvent their normal update/review process when you use Expo. One day they could just decide to reject all React Native apps. I really don’t think they would because of the uproar it would cause. I’m simply pointing out that it’s their platform and they would have every right to do so. So is it all about investing for your own gain? So far, I’ve presented all this from kind of a cold, heartless investor perspective: getting the most for your time. But, that’s not the whole story is it? Life isn’t all about me. Life isn’t all about us. I want to invest in platforms that increase opportunities for others. Personally, I really hope the next friggin’ Mark Zuckerburg isn’t an ivy-league dude. Wouldn’t it be amazing if instead the next huge success was, I don’t know, perhaps a young woman in Nairobi or something? The thing is, if owning an iPhone is a prerequisite for building apps, it dramatically decreases the odds of something like that happening. I feel like the Web really is the closest thing we have to a level playing field. I want to invest in and improve that platform! This quote really struck me and has stayed with me when thinking about these things: If you’re the kind of person who tends to succeed in what you start, changing what you start could be the most extraordinary thing you could do. – Anand Giridharadas Thanks for your valuable attention ❤️. I’ve presented the facts as I see them and I’ve done my best not to "should on you." Ultimately though, no matter how prepared we are or how much research we’ve done; investing is always a bit of a gamble. So I guess the only thing left to say is: I’m all in. Big thanks to Addy Osmani for his super helpful feedback on this post.

Installing web apps on phones (for real)

If you're building for the web, you've likely heard the term "Progressive Web App" by now. Its definition is a bit vague and somewhat contentious perhaps, but generally the idea is that it's a web page that has "taken all the right vitamins" so it can behave more like an app you installed from the app store. It starts as a normal tab in your browser and if it has all the right stuff the browser will prompt visitors if they'd like to "Add to Homescreen", which up to this point has largely equated to being a glorified bookmark. Once opened from the Homescreen it hides the browser UI and appears as its own app when switching apps, etc. This is huge! If you're not familiar with the challenges currently facing native app developers, here are a few: It's nearly impossible to break into the small group of 5 - 8 apps users use regularly. On average, in the US, smartphone owners install zero new apps per month. You have no control over the approval process. Paid apps obviously require paying fees to the app store. PWAs are an attempt to hit the sweet spot balance between web and native. Fast, native-app-like experiences without the list of problems above. The simple act of opening one of these Webpages With Special Sauce™ means you've already downloaded "the app" so there's no need for an App Store run-around where you ask users to download a 50mb binary over their flaky mobile connections just so they can read your web content. Arguably, it's really more like "saving experiences" you're currently having in your browser for easy access later, even if you're offline. This ability for a web page to graduate to becoming a full app-like experience appears, by all measures and case studies that have come out so far to be pure magic juice for businesses who care about you know, those silly little useless metrics like conversions or retention or whatnot. My professional interpretation of the data I've seen on this so far is: OMG. But don't take my word for it, watch this instead. PWAs are the entire reason I switched to Android after being on iOS for 7-some years. The blog post I wrote about why I switched apparently struck a nerve. It had 80k+ uniques in the first two weeks and led to several response posts by folks like Jeremy Keith, et. al. I'm not saying this to brag (well, maybe just a little), but the point is, this stuff is a big friggin' deal and we have good reason to be excited about it! For me personally, building web apps that feel like native apps is something I've been trying to do for ages. Exhibit A: this screen cap I made about something I built for this seven years ago! Let's just say the concept has just always made a lot of sense to me. However... The implementations have historically fallen short Of course, building app-like websites are hardly a new idea. This is how Steve Jobs originally proposed folks build for the iPhone! But as I explained in more detail in my post. I don't believe the incentives made sense for platforms to do this kind of thing at the time. I'll spare you repeating my other post, you can just go read it if that sounds interesting. The time has come PWAs have already come a long way even without the changes I'm discussing here. They're already solidly in the "very likely a much better investment than building native apps" category for most companies considering building mobile apps. But, as awesome as PWAs are on Android, they still have had a few annoyances that keep them from being truly "first class citizens" on the platform. For example: PWAs added to Homescreen are not listed in the "all apps" panel on Android. This also means it isn't possible to have PWAs that are "installed" but not on a Homescreen somewhere like you can with other Android apps. Once it's added to your Homescreen, the app developer can't later decide to change the name or the icon. Changing the published Web App Manifest has no impact on previous "Add to Homescreen"-ed apps. When you migrate from one Android phone to another, as I recently did when I got a Pixel, my "installed" PWAs lost their full-screen-ness and didn't run in standalone mode until I removed and re-added them. Even though App Store discovery has proven very challenging certainly users have been trained to look for apps in app stores. So if you tell your users you built an app the reaction is probably going to be to go search for it in the app store. But, as of right now, they're not listed in app stores (though Microsoft is changing that). The phrasing of "Add to Homescreen" is pretty weak. It doesn't imply that you're doing more than adding a bookmark. For better or worse we've taught users that apps are to be "installed". As a developer of PWAs, I can tell you that the phrase "Add to Homescreen" feels like it's selling what I built short. I know users won't quite get this, whereas they certainly know what "installing an app" means. An installed native app can "own" a "url scope", which means its possible to create a link that opens an installed application. This may not seem like a big deal, but say you're building a password-less login system where a magic login link is emailed to you that logs you into the application. As it stands right now, there's no way for me to tell the OS to open that link in the installed PWA, instead they'll always open in the default browser. Web push notifications as implemented right now, come from Chrome, not the PWA. This also means that if you block notifications, it's kind of all or nothing. Because the OS is blocking Chrome from notifying users. Stuff like storage usage and battery consumption get bundled in with Chrome rather than being presented as isolated apps to the user. So here's the interesting thing about all these problems: most, if not all of them, stem from the fact that "installed PWAs" are not truly apps in the eyes of the operating system! That's what's changing! Soon "installed PWAs" will actually be APKs, as in real Android apps will be created and side-loaded into the OS by Chrome. This means they won't be websites that have been "Add to Homescreen"-ed they'll be actually generated and installed on the fly, and with the language to match. It was rather subtly announced by Paul Kinlan, please watch his talk from Chrome Dev Summit the specific part I'm referring to starts right here. It seemed like not a lot of people even caught it. Personally, I erupted into spontaneous applause (in fact, you can hear me on the video right here), but I think, given that this was the last talk of the conference and a lot of folks were kinda zoning out I felt like perhaps the implications of this were a bit lost. So, let me be clear: I think this is a BIG FRIGGIN' DEAL for the web and worth being very excited about! What happens? Beware, I'm told the way it's implemented as I will describe below is not its final state. Apparently the whole "side-loading" bit will be smoothed over and be more seamless. There's a list of bugs related to all of this stuff, you're curious. It's not landed yet, it's currently behind a flag in Canary so details are sure to change, but you can try it today if you've got an up-to-date copy of Android and Chrome Canary installed. 1. Turn on the flag Go to chrome://flags and enable "Improved add to Home screen": 2. Open a sweet PWA and install it... I'd recommend something like paperplanes.world and select "add to Homescreen" from the menu. You'll see this: Note: The language is all about "installing this application" because you're actually side-loading an Android APK here! Next you get this "app installed" language: And now, you've actually got an APK installed for this web app! Which means it's also listed in my "all apps" view: You can even view app info like other apps: 3. Why does any of this matter? A platform with over 1 billion monthly active mobile is working to make web apps first class citizens on that platform in every way they can. This is a huge deal. But, to be clear this isn't about Google and Chrome, sure the term Progressive Web App may have been coined and promoted by Googlers. But, don't confuse this with just being a Google Thing™. I'm pretty sure the folks at Microsoft, Mozilla, and Opera would take that personally. Friggin' Steve Jobs promoted the basic idea. Microsoft has committed to listing PWAs in their store! Which is actually more than Google has committed to doing though, if I were a betting man, I'd say this little APK tweak is a step they're taking in that direction. But regardless, the point is this isn't about Google, it's a Web Thing™. Right now the Google folks are definitely among the group pushing the envelope with this concept the hardest. But none of this would get me excited if it were not based on extending the reach of standards-based web applications onto their host systems. As someone who's been working in this space for a long time I'm super excited to see metrics support and economic forces all finally seem to be aligning behind this idea, its time seems to finally have come. It's a good time to go all in on the web. I can't wait to see what the next few years bring. Personally, I feel like the web is well poised to replace the majority of apps we now get from app stores. Oh, and for those of you worried about Apple, don't be. They don't like to be the laggard and there are positive signs that they're coming around to the idea. I think the best way to make that happen is to building amazing web experiences that make choosing to invest in building native apps for App Stores seem arduous and unnecessary. Apple certainly don't want Safari to be the new IE, so watch them closely and help speak out where you can. If you want to (nicely) disagree with me or if you just think there may be something to this whole Web Thing™ I'm @HenrikJoreteg on twitter you can hit me up there.

Why I switched to Android after 7 years of iOS

Monday of last week I was all excited. I had just gotten the green light to start prototyping a new Progressive Web App for a client I've been working with. I pulled out an older Android phone that I keep around for development. Then I also got my sleek, new, shiny iPhone 6s out of my pocket, with its smooth curves and speedy OS. But as I looked at my iPhone I was kind of bummed out. I realized that this slick piece of Apple hardware was less capable as a platform for web applications than my dusty old Android dev phone. At that point I knew I was over iOS. So, instead of opening my text editor I placed an order for a Nexus 6P and signed up for Google fi phone service (which is awesome, btw). Just like that, after 7-some years, bye bye iOS. What!? What’s wrong with iOS? Remember the original iPhone announcement? Where Steve introduced the amazing combination of a mobile phone, an iPod, and an Internet communications device. I don’t know about you, but the idea of having a fully capable web browser in my pocket was a huge part of the appeal. That’s never changed. Of course I don’t know the full backstory, but it sure seemed like the original plan for 3rd party developers on iOS was to have us all just build apps using the web. Safari added support for making web apps that could be added to your home screen as an icon and by including a few magical <meta> tags you could use to create something that could sort of be “installed” to your home screen. Then, when you opened it would run in “standalone mode”. These were, in many ways, the original “Progressive Web Apps” and this was sometime around 2009! Think about it… They started in the browser, but then you could kind of upgrade them to “home screen status”. When you ran them from the home screen they’d open with a splash screen and without any visible browser UI. You could pick a loading screen and app icon. You could even even pick from a few different status bar colors! I don’t know whether or not this type of app was actually intended to be the primary mechanism for 3rd party dev to build apps for iOS but regardless… it was way ahead of its time. Unfortunately, the web platform itself wasn’t quite ready for the spotlight yet. It was sort of possible to build web apps that looked and performed like native apps, I was trying to do this 6 years ago using David Kaneda’s awesome, jQTouch lib. Hilariously, the corny little demo video that I posted, led to a call from David and almost got me a job at extjs right as they were rebranding to Sencha and starting to build Sencha Touch. But the story for offline was terrible. But anyway, as it turned out, the capabilities of the web on iOS were not quite enough to satiate ravenous developers. So developers were left clawing for the ability to build stuff that ran natively on the device to give them better performance and deeper API access. Enter the iOS SDK and App Store Apple made what turned out to be a really smart business decision: they released an iOS SDK and an App Store and the rest is history. First, I was excited about “apps” just like everyone else seemed to be. Just think, here we had been busy building “applications” when we really should’ve been building “apps” all along! Who knew?! ;) Anyway, I quickly found myself hunting for the best apps and switching to whatever bank, social networks, and other services had the best iOS apps. I bought a book on iOS development and built a hello world or two. My old co-worker Ryan Youngman made iSaber to let you swing a fake lightsaber at your friends with your phone. Every developer I knew was talking about iOS development but at some point the fun of all this iOS stuff dried up. Seeing the hoops you had to jump through to ship an app on iOS didn’t seem right. How quickly developers traded away the wide-open-spaces of the web for a walled castle with a monarch enforcing a 30% tax. So, I decided to focus on building “installable web apps” for iOS instead because surely, the web would catch up. However, this became problematic Despite the popularity of native apps the original idea of these standalone installable web apps has continued to be supported for new versions of iOS. But, they didn’t fit into Apple’s business model! The App Store turned into a huge business, the term “app” was going mainstream, and every business suddenly felt they needed to have their own “app” whether they had any users or not. As Apple’s app business took off these capabilities very clearly, very quickly, and somewhat unsurprisingly were deprioritized. The end result, for those of us still trying to build installable web apps for iOS was that with nearly every new iOS release some key feature that we were depending on broke. I’m talking about stuff that QA should have caught, stuff that if anybody at Apple was actually building apps this way would have noticed before they released. One quick example that bit me was how they broke the ability to link out to an external website from within an app running in “standalone” mode. target=_blank no longer worked, neither did window.open or anything else I could think of. So now since our “standalone” app didn’t have a URL bar or back button it would simply take the user to the link they clicked within the same full-screen web view with no way to return to the app! The only way out was forcibly quitting the app (hopefully the user knew how to do that). We were running a chat product at the time, so anytime someone pasted a URL into chat it was essentially a trap. These sorts of issues continued to happen release after release. Soon it became obvious that while you can sort of build these types of apps on iOS you can’t really depend on them not breaking with the next update. The message from Apple seemed clear: web apps are second-class citizens on iOS What of Android? I didn’t care much at the time, but somewhere in the middle of all of this, Android appeared on the scene. It promised to be a more open alternative as a mobile platform. It was a collaboration between several big companies, it was their attempt to essentially fight off the fruit-company-comeback-kid-turned-gorilla and its Mighty Joe App Store. It started gaining traction, but its web experience at the beginning was quite sub-par. Fast-Forward five years… People are somewhat burnt out on Apps. The vast majority of developers building native iOS apps never even make back their expenses. We knew this in 2013. A few games are still making money, but that’s a lottery. Meanwhile, there are over 1.4 billion active Android users. Android switched to using Chrome as the default browser. Chrome, Opera, and Firefox have added features to allow building actual app experiences via the Web. And here I am… switching to Android. So why Android? Isn’t it just more of the same? Yes. It is. Android itself bores me, honestly. There’s nothing all that terribly new or exciting here.’ save one very important detail… IT’S CURRENTLY THE BEST MOBILE WEB APP PLATFORM What do you mean?! Doesn’t Safari run my JS is faster? Most people when they say this are referring to this post by Jeff Atwood (a.k.a. codinghorror), which I wrote a whole response post to, if you’re interested. So yeah, Safari runs my JS faster, but guess what… most of your users won’t have a shiny new iPhone 6s, and as I’ve said before, betting on desktop-like performance on the mobile web, or sending huge frameworks like Ember to a mobile device probably isn’t a great idea. With performance, there is such a thing as “good enough”. It wouldn’t matter if Safari ran JS 50x faster! The only thing that matters is whether my app runs fast enough. Beyond that, as a user, I don’t care. As it turns out, it’s possible to write web apps that run at 60fps even on older, crappier hardware. But, all that aside, note this: I said “better app platform” not faster JavaScript runtime. So why not just use Chrome for iOS?! As I started tweeted about switching I was surprised to realize that many people don’t know that Chrome, Opera, and Firefox for iOS all just using WebKit web views under the hood. In fact, apps that include a different browser engine are a violation of Apple’s terms of service. They're just different UIs on the same browser engine. But isn’t WebKit getting better? Yes, it seems like they're picking up some momentum recently. But, there’s a whole lot more to it than just what happens in the browser window. I want the ability to create app-like experiences on the OS with web technology. Very little seems to be happening in that regard as far as I can tell. Let's look at Apple and WebRTC A few years ago when I built SimpleWebRTC and the first version of Talky.io. I seem to have been one of the early web geeks to get really excited about WebRTC (the browser web technology that now powers Google Hangout video calls). Anyway, I managed to figure out how to build one of the first, possibly the first multi-user, peer-to-peer, video calling WebRTC app on the web that worked with more than 2 people and worked between Chrome and FireFox. This was my first experience with Apple lagging behind in implementing new web APIs. Though Chrome and FireFox were both actively implementing and excited about WebRTC, there was not a peep from Apple. iOS still hasn’t added WebRTC support to this day. Though, they’ve apparently been hiring WebRTC engineers of the Safari team. So here’s hoping… But it kinda makes sense, right? Why would they? They’d rather you use FaceTime, right? They seem fine with improving the browser engine, but seem very slow to do anything that involves increasing the web’s reach in the OS. Anyway, we shipped Talky.io as a web app that worked in Chrome and FireFox and eventually @hjon built an iOS app for it too. But the thing that blew my mind was one day I just downloaded Chrome on Android, opened it to Talky.io and sure enough… IT JUST FRIGGIN’ WORKED! Since then I’ve been paying much closer attention to what’s happening in mobile Chrome and it’s very impressive. Meanwhile on Android During the last couple of years, a few very bright, very persistent, and idealistic developers (many of them at Google) who believed in the web have been at work pushing for, and implementing new web standards that fill the gaps between native and web. Incredibly cool new stuff is coming, like: WebBluetooth (yup, talking to bluetooth devices from JS on a webpage) WebNFC is coming too, apparently These things are going to blow the roof off IoT stuff (but that’s a whole other blog post). Just type chrome://flags in the URL bar of Chrome for Android and read through all the stuff that's currently in the works. It's amazing! Anyway, in the past couple of years these fine folks have built a feature that has me more excited than I’ve been by any web tech for a loooooong time: ServiceWorker and the concept of Progressive Web Apps. I believe that the introduction of ServiceWorker and Progressive Web Apps on Android is the most important thing to happen to the mobile web since Steve first introduced the iPhone. Why?! Because, for the first time, we have a mobile platform with a huge user base that lets me build a web app that is treated as a first-class citizen by the platform! (note: yes, I’m aware there have been other attempts to do this, but none of those had 1.4 billion active users.) These folks finally gave the us a platform where web apps were first-class citizens! And to be clear, I’m not just talking about a way to put a glorified bookmark on the home screen. I’m talking about a way for us to build web apps that are indistinguishable from native apps. The terms that’s sticking for these types of apps are “Progressive Web Apps”. In fact, I think Progressive Web Apps (PWAs) actually have a huge leg-up on native apps because you can start using them immediately. You don't have to jump to an app store and wait a minute or two until some huge binary is done downloading. They’re just web apps, they have URLs, they can be built to load super fast. Because… well, we’ve been optimizing load time performance on the web for a long time. There’s just so much less friction for users to start using them. Just think what that would do to your conversion numbers! Because of the improved on-boarding experience I believe that businesses targeting Android users should be strongly questioning whether they should be building native Android apps at all. So what are Progressive Web Apps anyway? Unfortunately, for some reason Google has managed to teach a generation of devs the words “Polymer” and “Angular” while the vast majority of web developers that I meet and talk to today have still have ZERO idea what ServiceWorkers or Progressive Web Apps are. Some of this is because of the newness of it all, and some of this improving recently… but sheesh… I hope this changes. You can think of a a progressive web app like this: It’s an app written in HTML, CSS, and JS that can completely masquerade as a native app. This includes: Living on the home screen Existing in the Android “app switcher” as a separate app (not as part of the browser app). True offline behavior… meaning when you tap the app icon… it will open regardless of current Internet status. The ability to run in the background and triggering OS-level notifications, even when the app and browser is closed. Instead of starting as a useless web page with a “please install our app” banner, these apps starts life running as a tab in your browser. Then progressively they become more installed/integrated into the OS. At first, it’s really no different than any other website you visit. But, then if you return to that same website/app in your browser again, the browser itself will subtly ask the user if they’d like to add it to their home screen. From this moment on it’s indistinguishable from a native app to the user. Also, if you build these correctly there’s usually nothing else the user has to download or wait for at all. This means that adding it to the home screen is effectively an instant app install. Again, imagine what that’ll do to your conversions? Eh? (no, I’m not Canadian) Luckily, we don’t have to entirely guess about the business impact. We actually have some real data from a certain $20 billion dollar online retailer in India called FlipKart, who did launched a PWA and have shared some of their numbers. Key highlights from FlipKart’s experience: 40% returning visitors week over week +63% conversions from Home screen visits 3x time spent on FlipKart Lite That data came from Alex Russel’s recent Fluent Keynote on what’s next for mobile. I encourage you to watch and share it with product managers and leaders at your company. It does a great job of explaining the how/why of Progressive Web Apps. For related reading check out: Addy Osmani’s Getting started with Progressive Web Apps Mozilla’s service worker examples at: ServiceWorke.rs FlipKart’s original technical post about their PWA Jake Archibald’s Offline Cookbook Aditya Punjani's post on how they built FlipKart lite So what does this all mean for us? We, as web developers, can finally build screaming fast, fully offline-able, and user-privacy-protecting apps that work cross-platform without the need for any friggin’ App Store taxes, approval processes, or doorslamming users up front with “please install my app to use this service”. What about iOS support? Well, the beauty of it is, iOS user can still use your web app even if service worker support doesn’t exist. They just don’t get the extra goodies, like offline and push notifications. But you could also bundle the app with Cordova and use the Service Worker plugin, that would, in theory let you use the same code to do those things but bundled up as an iOS app. Why should I care? React Native exists now and solves the same problem. Personally, I actually kind of wish tools like React Native didn’t exist. Stay with me, let me explain. React Native is an amazing and very impressive tool that lets us use our JS skills to write native iOS apps. But as I’ve been saying… I don’t think we should be building native apps unless we absolutely have to. The end result of React Native is that because it exists and because its largely aimed at web developers we now have web devs flocking to build native apps just because they can! I fear that this undermines our ability to use our collective bargaining power to encourage Apple to implement support for Progressive Web Apps. To be clear, I completely understand why it was created and I have a lot of respect for the technical achievement it represents, and the developers behind it. I just don’t want us to stop pushing Apple to improve web support. In summary So, all this said, these things led me to finally exercising the only voting power I have as a consumer… I took my money and left. I don’t see this as switching to Android, I’m simply switching to the best mobile web app platform available today. The web is the only truly open platform we’ve got. It’s the closest thing we have to a level playing field. This is why I’m focusing all my efforts on building Progressive Web Apps… I hope you’ll do the same. — I’m @HenrikJoreteg on twitter if you want to nicely tell me all the ways in which I’m wrong. See ya on the Interwebz <3

The viability of JS frameworks on mobile

Whether I like it or not, not everyone using my web apps will be running iOS 9 on an iPhone 6S or a Nexus 6P and connecting via super-speedy wifi. The reality is often anything but that. 3G connections and older hardware is often the norm. Google reports that there are 1.4 billion active Android users. Many of them, will no doubt, be running less-than-top-of-the-line hardware. And, if you read stuff like Jeff Atwood’s recent post on Android performance you may feel that things are pretty bleak for the mobile web. A few notable pull quotes from that post: In a nutshell, the fastest known Android device available today -- and there are millions of Android devices much slower than that out there -- performs 5× slower than a new iPhone 6s, and a little worse than a 2012 era iPhone 5 in Ember. How depressing. We've done enough research to know this issue is not really specific to Ember, but also affects Angular and most other heavy/complex JavaScript on Android. Why? Perhaps the “heavy/complex” part is the problem? Continuing… This is becoming more and more of a systemic problem in the Android ecosystem, one that will not go away in the next few years, and it may affect the future of Discourse, since we bet heavily on near-desktop JavaScript performance on mobile devices. That is clearly happening on iOS but it is quite disastrously the opposite on Android. Ok, there’s the state of the mobile web ecosystem, per Atwood. He ends with this line: I am no longer optimistic this will change in the next two years, and there are untold millions of slow Android devices out there, so we need to start considering alternatives for the Discourse project. Bummer, right?! Let’s all go home. At least there’s React Native so we can all pretend we’re building web apps while actually writing native apps ;) So, is the mobile web a dead-end for apps? Hang on a minute… If you didn’t catch it. The “Discourse” app Atwood is referring to is the app he used to publish the post about Discourse. It’s a forum app of sorts. So meta! Anyway, when I realized this, I open the network panel of dev tools on that Discourse page and there was 659kb of JS transferred across the wire (that’s the gzipped size). In my opinion, that’s the same as forfeiting on mobile before you even start. I think we have to do better than this to be viable on mobile. I’m not blaming or even trying to criticize Atwood or Discourse here. A huge number of mobile web apps are built this way so this could really have been any number of sites. My question is simply: are all these heavier tools/frameworks even viable for mobile use? I’m not convinced they all are. Let’s look at some research The fine folks at The Filament Group published some research last December about load time performance of the TodoMVC app of 5 popular web frameworks. Obviously, TodoMVC is a bit of a contrived example and may not be indicative of a real-world app. But, it’s a great fit for this type of research because it will include all the base assets for a given framework, and at least hopefully, the best practices as well. You may be thinking: ”this is load time performance, Atwood was talking about runtime performance!” Yes, Atwood was discussing runtime performance, I’ll get to that shortly. But, the user doesn’t care why they’re waiting, so load time is clearly an important part of performance too. The whole post is worth reading, but the summary of the research findings can be seen in their graphs: In my opinion, the data for Angular and Ember (the two options that Atwood mentioned) flat out disqualify them for mobile use. If you know that I co-created Ampersand you may assume that I’m flaming against other frameworks and trying to sell you on Ampersand. Nope. The last two apps I built actually didn’t contain any Ampersand code at all. If you’ve read my recent, excited post about Redux that probably doesn’t surprise you. I don’t care what you use, beyond how it affects how I experience your app as a user. Tools are merely a means to an end. Ok, I lied. I do sort of care what you pick. But, only because I don’t want your experiences as a developer lead you to think the mobile web isn’t viable just because sending a megabyte of JS made the app slow. It’s not game over Maybe the mobile web is fast enough and we just need to stop pretending we can get away with ineffeciencies that we don’t feel on a desktop. I think we need to be much more minimalist from the start. The ever brilliant Dominic Tarr once said this: if you want to write fast software, use a slow computer — Dominic Tarr (@dominictarr) August 8, 2015 How many web developers, to this day, don’t test on a phone locally when building apps? This isn’t just about small screens, we need to assume that we’re building for weaker, slower computers. I think we need to bake mobile into our dev workflow, not just as some final pre-launch check. If you’re curious, I use this setup while developing. Regardless, even though phones may continue to get faster and faster, I think it’s myopic to assume we can just ignore current speed problems in hopes that they’ll be resolved by faster hardware. Although, phones may continue to improve, I don’t think phones should be our only runtime target. As this whole Internet of Things, um… thing becomes more of a thing (wow, that sentence was epically terrible #leavingitanyway). It seems likely there will be a some other platforms we’ll want to write for that are not phones. Things like watches, TVs, VR, and small computers hooked to to large displays. That’s nice, Henrik. So what do you propose we do? First, as I’ve written before, I think all the HTML we can infer from the URL alone should be rendered at build time to static HTML files. As it turns out, browsers, even “slow” mobile ones are pretty friggin’ fast at rendering HTML sent from the server. Secondly, I think we need to do more with less, let me explain… Much of the hoop-jumping in Backbone, Ampersand, Ember, and Angular have to do with properly binding state to the UI. They certainly do more than that, but even many of the data features, on closer inspection have to do with supporting those binding capabilities. As a result, they all ship with an event system of some kind, and most have a way to create derived/calculated properties that can also be observed. You can certainly use React as a view layer for any of those, but one of the big wins with React is that at the point where you can do a cheap re-render it removes the need for many of those other features that add complexity to your system. As a simple example, if you’ve got a Backbone.Collection with models in it, how do you go about rendering the length of that collection to the UI using Backbone paradigms? Getting the value in the template initially is certainly easy if you know it at the point when it’s first rendered. But, how do you bind that value so that any change in the collection will also update that length that you’ve rendered into the DOM? You could set it up to re-render on all add/remove/change events or, you could create an observable property somewhere that did the same and then track that as you would any other property. That total would be updated with any changes, and then, you’d bind it to the view. But all this feels a bit silly. If you’ve ever inspected an instance of a Backbone.Collection you know that collection.models is simply an array of Backbone.Model’s. An Array already has a value for length, it’s called .length **sigh**! Contrast that to React or anything else that allows us to inexpensively re-render our whole app at will (the promise of React). When we have that, we can just use that length property directly in our render method in the individual component that cares about it. Now whenever we have any change in any of the state in our app we re-render. So now it becomes this simple: React.createClass({ render: function () { return <span>{this.props.species.length}</span> } }) Or, if using the new stateless functional components in React 0.14, with ES6 destructuring, and assuming species is our array of objects here it gets downright beautiful. The entire component could be written as: var Aquarium = ({species}) => ( <span>{species.length}</span> ); Aesthetics aside, just think how much less stuff we need: We no longer need to register multiple event listeners on the collection (less code, less memory use). We no longer need to define another property that we have to compute that creates an observable duplicate of what we already have with .length (less code, less memory). Shoot, we may not even need the collection or any of its models to be observable at all! The collection could be a plain ‘ol JavaScript array containing plain ‘ol JavaScript objects instead of custom created observable models (a lot less code, less memory, less computation) We no longer need many of the features in those heavy frameworks at all. React then becomes the biggest pill you have to swallow. In my simple tests with React, and React-DOM 0.14, built with webpack and gzipped you end up around 37kb. For comparison, jQuery 2.x weighs in at about 29kb min+gzip. So, really, we’re doing pretty good and my hope would be that eventually, much like happened with jQuery, many of the core features of React would simply be part of browser API. Anyway, now rather than needing observable models, observable collections, the ability to subscribe to changes on derived properties and all that jazz, we can probably get away with an architecture that works more like this: A single application state object as a JSON-like structure of plain JS arrays and objects. A set of mutator functions that represent all the ways that state can be changed to be used any time there needs to be a change in the state tree (you could follow immutability principles here too, if you’d like and just make sure you replace anything you change). Re-render the whole app any time these mutator functions are done. I’m simplifying a bit, but that’s pretty much how Redux works. Read my Redux post for a better overview. But, you don’t need Redux to implement a simple pattern as described above. Oh, an by comparison Redux weighs practically nothing at approximately 2kb. The point is, that by leaning on React for DOM syncing we can dump a bunch of stuff. Imagine how much less code it takes, and how much less work is required of the browser to deal with a simple set of objects and arrays and a smart render call. As opposed to shipping a class system, an event system, a templating system, a custom set of observables types, a DOM library like jQuery (required by some of them), all the extra app-specific code you’d write describing those models and collections and all their properties. Instantiating models and keeping all their internal state and caches those models use to enable their capabilities will require more memory and computation. The processing and memory required to do value comparisons (also known as “dirty-checking”) to see whether a change needs to be triggered. There’s just so many more layers of code that have to run even for simple changes to the state of the app. The ability to re-render at will allows for a fundamentally lighter approach. Unfortunately, I don’t have any perfect example I can just line up next to the Ember runtime performance test Atwood links to, but it’s not hard to imagine that all these simplifications that flat-out reduce the amount of computation required, would also have a very positive impact on runtime performance. Also, to clarify, total file size may not matter so much as long as load time and runtime performance are still good. So, what’s next? Unless we want the web to be seen as a second class citizen on mobile, I think we need to address these performance issues. I believe the web can be fast even on mediocre hardware while providing a good developer experience. BTW, if you doubt the developer experience part watch Dan Abramov’s talk from React Europe. I certainly don’t have all the answers here and I’m not saying React + Redux is some kind of panacea. I just like what they enable. But so far, pre-rendering all known HTML and using the lighter approaches described here seems to be working well for me. I’ll keep sharing my thoughts as I keep building more stuff. Please do the same. Hit me up on twitter: @HenrikJoreteg or better yet, write a response on your blog to continue the conversation. Go go gadget web!

Reactive Programming

The term “reactive programming” may sound a bit cryptic, but if you’ve ever used a spreadsheet, you’re already quite familiar with the concept. When you’re simply viewing a spreadsheet, each cell isn’t necessarily showing you what was actually typed into the cell. Sure, some cells may contain simple values, but others actually contain formulas. The formula may, for example, describe that the cell should contain the sum of cells A2 and B2. But, when you’re viewing it you’ll see the computed value, not the formula itself. The real power of a spreadsheet comes from the fact that when you reference another cell from one formula the referenced cell could either contain a simple value or another formula. Any changes made to the inputs will flow through the entire spreadsheet. Each cell gets recomputed anytime any of its inputs change until the entire sheet is reflecting the changes that have cascaded through those formulas. What I’ve been describing with the spreadsheet is in fact reactive programming. Each cell is simply reacting to changes in the inputs. That’s the key distinction: instead of writing code that calculates simple values, we write code that describes relationships between sources of values. Once we’ve described how everything should react to the changes to its inputs, then the data flows through our functions until all the new values have been computed. So, why would we want to build things that way? What does it gain us? Well, let’s think of the alternative: If you have a table of values that you want to total up, you could certainly pull out a calculator, add them all up, and type the resulting value into the “Total” cell. Done! Easy. But, now what happens when the values change? Obviously, you’d have to repeat the exercise. This may be tolerable for a small table, but it would quickly become completely unmanageable if you’re dealing with, say, an amortization table showing 30 years of monthly principal and interest payments. Managing changing inputs in that type of system requires a difference paradigm. Let’s give a more specific example. Let’s say you want to build a mobile web app that is going to let people sort a list by touching and holding an item or swipe an item right and left to perform different actions on it. A browser gives us four different “touch” events we can listen for: touchstart touchmove touched touchcancel Where’s my touchhold event?! There’s no such thing. So if we want to build an app that triggers “sort mode” when a user has held their finger on an item for a half second. Think about what we’d have to do: Listen for a touchstart start a timer wait half a second Doesn’t sound too bad right? But wait… we also have to make sure they haven’t moved their finger too far, but we should probably still consider it a hold if they’ve moved just a few pixels. We also have to make sure we stop the timer if we get a touchend. So, now we have to also store starting position we have to compare that position with any subsequent touchmove events we have to continually calculate a distance moved from that starting point with any changes. As long as that distance is within our tolerance and the timer reaches half second before any touchend event… congrats, we have a “hold”. If we think about it, this is really spreadsheet-type problem, right? We’ve got some input cells that are simple values, some of which will be updated over time: starting X starting Y start time current X current Y current time We’ve got formula cells that will calculate as the time passes and inputs change: time elapsed distance moved We’ve got an is holding formula cell that uses the values from time elapsed and distance moved. Ultimately, all this touch-related logic could be broken out into its own library. Then, our program would only have to care about that is holding cell and update the interface accordingly. All the complexity of how those touches are tracked and measured would be contained within the library. This is just one example of types of issues reactive programming approaches help us deal with. Over time, you’ll find that much of the code we write is really just tracking and updating relationships between different variables. Recognizing that and learning to use reactive patterns allows us to more easily solve complex problems and break them down into small, manageable pieces. Then, just let the data flow. Note: I was asked to produce a writing sample explaining reactive programming for a technical writing contract. I'm posting it here because I thought it may be useful to someone (and yes, I "passed"). If you like this, or if you think I botched it, please let me know on twitter: @HenrikJoreteg