Ultimate Guide to Progressive Web Apps: Fundamentals

Welcome to Ultimate Guide to Progressive Web Apps blog series!

In this journey we are going to reveal what a PWA actually is and how its underlying technology (Service Workers) functions. No previous knowledge on this topic is required. We will start from scratch and go from newbie to profesional (with Workbox) step by step.

This first post explains the fundamentals, the core the concepts that everybody should be familiar with when they approach a progressive project. When you finish reading it you will have a firm grasp on this technology.

Definition

By now we all have an idea of what a (non-progressive) web app is. When we talk about web apps we talk about Front End, we talk about the client side, we talk about technologies that have to do with the browser. The concept comes from the Web 2.0, a dynamic web environment where the user can participative and collaborate. No doubt Microsoft contributed to make that interaction fluent by designing the XMLHttpRequest Web API in 2006. Thanks to jQuery we know such technique as Ajax and since it allows us to make HTTP requests without reloading the page this is what constitutes a web app as such.

However, believe it or not, it was Apple at the launch of the first iPhone in 2007 the one who first introduced the idea of “WebApp” as something in the direction towards progressive. Steve Jobs said:

“The full Safari engine is inside of iPhone. And so, you can write amazing Web 2.0 and Ajax apps that look exactly and behave exactly like apps on the iPhone. And these apps can integrate perfectly with iPhone services”.

Of course the App Store came later and they abandoned this conception in favour of native apps with SDK. But later on Google took that idea and moved it forward by proposing a new technology: Service Workers. We will talk about them later on in this post but for the moment just remember: the same way as a web app is only possible with Ajax, a progressive web app only exists thanks to a service worker.

Free eBook

Directives, simple right? Wrong! On the outside they look simple, but even skilled Angular devs haven’t grasped every concept in this eBook.

• Observables and Async Pipe
• Identity Checking and Performance
• Web Components <ng-template> syntax
• <ng-container> and Observable Composition
• Advanced Rendering Patterns
• Setters and Getters for Styles and Class Bindings

Download

That went smoothly, check your email.

So what makes an app progressive? When can we claim that we are looking at a PWA? Well, basically 3 conditions:

1) It loads fast enough for mobile networks. 2) It loads even when we are offline. 3) It is installable.

Whereas the first one can be accomplished by a web app the second can’t. The offline capabilities represent for us a threshold: once we cross it we enter the realm of progressiveness. But that is not all. Think for a moment about the third one: PWAs are installable! Like a native app or a desktop app. As a matter of fact a PWA is cross-platform and this is really amazing: only with our web code you are able to create an application that not only can be rendered on all browsers but it can also be installed and accessed the same way as an app from Android, iOS, Windows, macOS or Linux. And this achievement doesn’t need of any compilers such as Cordova or Electron.

Installation

To make a web app installable it should:

1) Use the HTTPS protocol. 2) Register a service worker. 3) Implement a web app manifest.

The secure HTTP communication is a logical requirement. We do not want to install anything that is not signed by trusted partners and free of attackers interference.

The service worker is the key to everything but we will talk about it soon enough.

And the web app manifest is really just a JSON file that defines the parameters of our installation. We include it in our HTML.

<link rel="manifest" href="/manifest.json">

We will take a deeper look into this in the next blog post.

But how do I install? Despite there are ways of uploading a PWA to either Google Play Store, Windows Store or iOS App (although this one is discouraged by Apple) this kind of app is typically installed through the browser.

If you are using a desktop machine you will need to navigate to the app domain with Google Chrome to install it. And how do I know if I am in front of a PWA or not? Easy: since Chrome 76 there is a install button in the browser’s address bar.

If you click on it you will get the installation prompt.

If you are on an Android device and you land with Google Chrome on a PWA you will automatically get the Web App Install Banner, also known as Add to home prompt since there is an “Add to home screen” button in it.

But in the near future that button is meant to be called “Install” since this text is more engaging for the user.

The reason we get those prompts is because in Chrome’s Web API there is a beforeinstallprompt event whose prompt() method is triggered automatically on Android. And the cool thing here is that this allows us to create our own install button.

Unfortunately iOS devices rely on Safari. This browser lacks the beforeinstallprompt event and therefore the installation in this case is a little different: we do not get any prompt. We need to click first on the Share button.

Compare the installation on Android and iOS respectively:

As you can see they are just different paths to the same goal.

WebView

When you launch a PWA it accesses a technology that is common to any mobile or desktop system: the WebView.

Imagine that you open a social media app on your device. There is interesting news about a topic that you like. To read about it you need to click on an external link. And what happens then? You open a web site without getting out of the app. In that case you are not opening an stand-alone browser but something called in-app browser.

An in-app browser renders web content by using a Native WebView. A PWA follows the same principle. You can think of a WebView as a browser without the browser whose only purpose consists in rendering. However for non-displaying browser features the view needs to access the system’s browser engine.

That being said, you need to be aware of the fact that different manufacturers have different specifications and therefore the WebView API varies.

Browser engines are essentially virtual machines made of 2 parts:

• Rendering engine.
• JavaScript engine.

When a WebView needs advanced rendering or JS functionality it goes to the browser engine and asks for that. This is the actual process that makes PWAs slower than native apps and that’s why the Chrome team is trying to replace WebViews with a new technology called Trusted Web Activity (TWA). This new API can check cryptographically that the app owner is also the content owner. It is faster than WebView and it has a complete Chrome API but it does not take Web Components yet. TWAs are also the official way of shipping your app to the Google Play Store.

Features

As you may know already the most important browser engines are:

• Chromium for Chrome, with V8 as JS engine.
• WebKit for Safari, with Nitro as JS engine.
• Gecko for Firefox, with SpiderMonkey as JS engine.

Since Gecko is not bound to any operating system we will only care about Chromium and WebKit.

Chromium has a great support for PWAs. Just to mention some of the most interesting features:

• Offline capabilities.
• Installation through prompt.
• Push Notifications.
• Background Sync.
• Persistent Storage through IndexedDB.
• Web Payment.
• Web Share.
• Access to camera.
• Access to audio output.
• Geolocation.

In contrast, WebKit has some limitations:

• Offline capabilities with a Cache Storage quota limited to 50MB for Service Workers.
• No beforeinstallprompt event.
• Only partial support for manifest.json.
• No Push Notifications.
• No Background Sync.
• No persistent storage and after a few weeks all your PWA files will be deleted.
• The access to the camera is restricted to photos only.

Nevertheless the WebKit team is working on a full support for the Web App Manifest and also considering Push Notifications.

You need to be well aware of these limitations before you decide the most apt technology for your project. For instance if you are aiming for mobile but you do not want to code native apps because it involves a duplication of the source code (Java Kotlin + Swift) apart from PWAs you can build natively-compiled apps (e.g. with Flutter, React Native or NativeScript) or hybrid apps (e.g. with Cordova or Ionic + Capacitor). These other 2 options demand a compilation step but at the same time offer better access to the hardware capabilities of the device.

Service Workers

The reason why PWAs load fast is because they follow the App Shell Architecture. An app shell is a minimal HTML, CSS and JavaScript required to power our user interface. You can think of it as the PWA’s substitute of the SDK in a mobile context. Take a look at this:

This screenshot belongs to the Google I/O 2017 conference. The first paint refers to the moment when the first pixel changes on the screen. On the first contentful paint there is a minimal interface but no dynamic information loaded yet, maybe just a spinner. It is only on the first meaningful paint when page’s primary content is loaded. Then we have a visually ready stage: the page looks done but it is not done yet. Only when all the processes have finished the app has reached its time to interactive.

So to make things clear: the First Contentful Paint corresponds with the App Shell and the Time to Interactive is the moment when we can show the prompt.

This PWA architecture relies on aggressively pre-caching this shell by using the magic of Service Workers. Thanks to them you can load the app even when you are offline.

JavaScript Threads

In order to understand service workers we need to keep in mind the fact that JavaScript is a single-threaded language.

Single-threaded is the opposite of concurrent. There is only one global execution context, also known as “thread” or just “context”. That technically means that you cannot run 2 or more bits of information at the same time. You have to do one thing at a time.

Javascript can run either on the client side (browsers) or on the server side (NodeJS). And depending on those contexts your code will be relying on different global objects:

• window in a browser.
• global in Node JS.
• self in workers.

NOTE: if you just want to get the global object regardless of the context you need to use the globalThis property.

Workers

Javascript Workers are scripts that run in a background thread separate from the main execution thread.

Since their global object is not window they do not have access to the DOM so if they do need some information from it the main thread and the worker thread will have to establish a communication channel via the Javascript MessageEvent interface. In other words, a worker can:

• Send a message through the Worker.postMessage() method.
• Receive a message through the Worker.onmessage property.

There are 3 types of workers:

• Web Workers. They have a general purpose: to offload heavy processing from the main thread. For example it would be a good idea to use them for image manipulation tasks.
• Worklets. Lightweight version of the Web Workers. They give access to low-level parts of the rendering pipeline (Javascript -> Style -> Layout -> Paint -> Composite). For example the PaintWorklet hooks into the paint rendering stage. This is what Houdini uses but it does not have full cross-browser support yet.
• Service Workers. Event driven workers that act as a proxy servers. PWAs key technology. They are only not supported on Internet Explorer (of course).

Service Workers Functionality

The idea of the service workers came from the Chrome team as a replacement for the deprecated HTML5 Application Cache. This specification was able to handle standard offline scenarios but not complex ones and nowadays it is deprecated.

But service workers go beyond the old AppCache. Of course they also deal fundamentally with files that are not meant to change in our app. We can pre-cache those files and speed up our performance in subsequent loads. But they also provide events for push notifications and background sync and they are meant to bring more in the future.

We can define them as requests interceptors. They can proxy any call either between the browser and the network or between the browser and the browser’s cache.

If there is information from a service worker that you need to persist and reuse across restarts, service workers do have access to the IndexedDB API.

A PWA is only installable if it uses secure HTTP communication because a service worker only runs over HTTPS and, also because of security reasons, it is re-downloaded every 24 hours or earlier in the case of an update. However http://localhost is also considered a secure origin for the sake of developing purposes.

If you want to explore the Service Worker API and see its cross-browser support there is n o better place to do that than Jake Archibald’s “Is serviceworker ready?” site.

Lifecycle

The service worker lifecycle ensures that the page (also called client) is controlled by only one version of the service worker at a time.

There are 3 lifecycle events:

1. Download: the service worker is requested through a registration.
2. Install: is attempted when the downloaded service worker file is found to be new.
3. Activate: it allows the service worker to control clients.

After the activation the service worker enters the Idle state. From here it can either be terminated to save memory or it can handle fetch and message events that occur when a network request or message is made from your page.

Let’s take a deeper look into the whole process.

Download

The first time we load our web page we need to register our newly created service worker. This registration takes place in the main thread so we can implement the code either directly in our index.html or in a separate file, let’s call it main.js. And we will say that sw.js is our file for the service worker.

This is actually the only code snippet you really need to be familiar with:

// ---> main.js
if ('serviceWorker' in navigator) {
  window.addEventListener('load', () => {
    // Register the service worker after the page is loaded.
    // Generally not before since this could slow down this loading step.
    navigator.serviceWorker.register('/sw.js').then(registration => {
      // Registration was successful so service worker is downloaded.
      // OPTION: registration.update();
      console.log(`Service Worker registered! Scope: ${registration.scope}`);
    }, error => {
      // Registration failed so service worker is not downloaded but just discarded. 
      console.error(`Service Worker registration failed: ${error}`);
    });
  });
}

As already mentioned the registration is automatically updated every 24 hours or every time the browser detects any change in sw.js after either a navigation or an event. However if you want to manually do that (update, re-download) you can call registration.update(). This could be useful if you expect your user to be operating on your site for a long time without reloading. In that case you may want to use hour intervals.

NOTE: remember that the service worker can only take control of the page if it is in-scope. Notice that /sw.js is located at the root of the domain. That means that its scope is the entire origin. If we had registered it at /scope/sw.js then the service worker would only be able to cache fetch events for those URLs that start with /scope/.

Install

After the registration the install event gets automatically triggered. Then we have the opportunity of pre-caching requests to some files that we consider as regular assets of our application: those files constitute the App Shell.

// ---> sw.js
var cacheName = 'my-site-cache-v1';
var urlsToCache = [
  '/',
  '/styles/main.css',
  '/scripts/main.js'
];

self.addEventListener('install', event => {
  console.log('Installing…');
  // OPTION: self.skipWaiting() instead of event.waitUntil()
  event.waitUntil(
    caches.open(cacheName)
      .then(cache => {
        // Precaching was successful so service worker is installed.
        console.log('Opened cache');
        return cache.addAll(urlsToCache);
      }, error => {
        // Precaching failed so service worker is not installed. 
        console.error(`Service Worker installation failed: ${error}`);
      })
  );
});

The installEvent.waitUntil() method gets a promise that tells our browser when the installation is successful. If we did not want to cache any files we would just write self.skipWaiting() and remove the whole waitUntil part.

This is how we implement the installation natively. But careful: every time we update sw.js we are updating the service worker and therefore we also need to update cacheName by hashing the name of the variable. We cannot perform this manual operation every time we change the file so we need to automate a build process for the service worker every time we make changes. Workbox performs this operation beautifully.

So don’t worry if you don’t fully understand the snippet. In practice we are going to develop sw.js with Workbox so the code will look totally different (easier). But we will talk about this in the next post.

Activate

If the installation is successful our service worker is ready to control clients but we are not quite there yet. In that moment the activate event gets triggered.

// ---> sw.js
self.addEventListener('activate', event => {
  event.waitUntil(
    caches.keys().then((keyList) => {
      return Promise.all(keyList.map((key) => {
        // Same cacheName that we defined before.
        if (key !== cacheName) {
          console.log('[ServiceWorker] Removing old cache', key);
          return caches.delete(key);
        }
      }));
    })
  );
});

You will also not need this snippet but it is good that you understand its idea. This will check the Cache Storage of our browser. If this is our first load the service worker will just simply get activated. But the fact that the service worker is activated that doesn’t mean that the page/client that called .register() (main.js) will be controlled already. For that we will need to reload the page unless you deliberately want to override this default behaviour by calling clients.claim() but this is not considered a good practice since it can be troublesome.

On a second load the service worker controls the client. And there is more: if you have made even a byte of difference on your service worker before re-loading you will be updating it and the browser understands this as a new service worker. As a result the updated service worker is launched alongside the existing one.

That is pretty interesting: the client can only be controlled by one version of the service worker at a time. In this case we would be playing with 2 service workers. The new service worker gets installed in the background while the old one is still active and if its installation is successful its activation is postponed by entering a waiting state until the old worker is controlling zero clients. For that we need to close all its windows (browser tabs), refreshing the page is not enough.

It is also worth mentioning that instead of closing tabs we could use the method self.skipWaiting() but we will see how this ambition can also be achieved by using the Chrome DevTools.

Still we need an additional step to intercept the fetch requests beyond the app shell but for that we will use Workbox Routing.

Conclusion

PWAs are a great choice not only for multi-platform projects but also for just web apps that demand a performance boost.

Initially all these concepts are a little difficult to comprehend but you can rest assured that in the next article you will learn by practice and then everything will become crystal clear.

We will talk about things like PWA audits, the Web App Manifest, caching strategies and debugging.

See you soon!