Building for the web gives you unparalleled reach. Your web application is a click away and available on almost every connected device-smartphone, tablet, laptop and desktop, TV, and more-regardless of the brand or the platform. To deliver the best experience you've built a responsive site that adapts the presentation and functionality for each form-factor, and now you're running down your performance checklist to ensure that the application loads as quickly as possible: you've optimized your critical rendering path, you've compressed and cached your text resources, and now you're looking at your image resources, which often account for the majority of transferred bytes. Problem is, image optimization is hard:
- Determine the appropriate format (vector vs. raster)
- Determine the optimal encoding formats (jpeg, webp, etc.)
- Determine the right compression settings (lossy vs. lossless)
- Determine which metadata should be kept or stripped
- Make multiple variants of each for each display + DPR resolution
- ...
- Account for user's network type, speed, and preferences
Individually, these are well-understood problems. Collectively, they create a large optimization space that we (the developers) often overlook or neglect. Humans do a poor job of exploring the same search space repetitively, especially when many steps are involved. Computers, on the other hand, excel at these types of tasks.
The answer to a good and sustainable optimization strategy for images, and other resources with similar properties is simple: automation. If you're hand-tuning your resources, you're doing it wrong: you'll forget, you'll get lazy, or someone else will make these mistake for you - guaranteed.
The saga of the performance-conscious developer
The search through image optimization space has two distinct phases: build-time and run-time.
- Some optimizations are intrinsic to the resource itself - e.g. selecting the appropriate format and encoding type, tuning compression settings for each encoder, stripping unnecessary metadata and so on. These steps can be performed at "build-time".
- Other optimizations are determined by the type and properties of the client requesting it and must be performed at "run-time": selecting the appropriate resource for client's DPR and intended display width, accounting for client's network speed, user and application preferences, and so on.
The build-time tooling exists but could be made better. For example, there are a lot of savings to be had by dynamically tuning the "quality" setting for each image and each image format, but I'm yet to see anyone actually use it outside of research. This is an area ripe for innovation, but for the purposes of this post I'll leave it at that. Let's focus on the run-time part of the story.
<img src="/image/thing" sizes="50vw"
alt="image thing displayed at 50% of viewport width">
The application intent is very simple: fetch and display the image at 50% of the user's viewport. This is where most every designer washes their hands and heads for the bar. Meanwhile, the performance-conscious developer on the team is in for a long night:
- To get the best compression she wants to use the optimal image format for each client: WebP for Chrome, JPEG XR for Edge, and JPEG to the rest.
- To get the best visual quality she needs to generate multiple variants of each image at different resolutions: 1x, 1.5x, 2x, 2.5x, 3x, and maybe even a few more in between.
- To avoid delivering unnecessary pixels she needs to understand what "50% of the user's viewport actually means"—there are a lot of different viewport widths out there!
- Ideally, she also wants to deliver a resilient experience where users on slower networks will automatically fetch a lower resolution. After all, it's all about time to glass.