Why HTML5 urgently needs an HTML adaptive images mechanism

After the recent kerfuffle about the draft HTML specification for a mechanism for adaptive images, and an excellent compromise suggestion by Florian Rivoal (Opera’s CSS WG rep), the mailing lists have gone quiet again.

(If you don’t know why we need such a mechanism, read Matt “Grrr” Wilcox‘s article Responsive images: what’s the problem, and how do we fix it?.)

We’ve needed such a mechanism for a while, but it’s particularly urgent now. Webkit has implemented a new non-standard CSS function called -webkit-image-set:

selector {
background: image-set(url(foo-lowres.png) 1x,
                      url(foo-highres.png) 2x) center;}

The changeset describes it:

The idea behind the feature is to allow authors to provide multiple variants of the same image at differing resolutions, and to allow the User Agent to choose the resource that is most appropriate at the time. This patch will choose the most appropriate image based on device scale factor.

(Read the email/ specification for -webkit-image-set.)

As far as I can tell, it will be in Safari 6 (OS X 10.8, shipping in July) and Mobile Safari for iOS 6, so we’ll be able to do adaptive background images but not adaptve content images.

Once that happens, I bet it’s 0.03 nanoseconds before developers convert <img src=”foo.blah” alt=”useful alternate text”> into an empty <div>s with adaptive background images using -webkit-image-set and (please!) a fallback background images for non-webkit browsers.

Why wouldn’t they? Retina screens will download massive but beautiful images, while users of lower-res phones save bandwidth by downloading smaller images instead of huge images and then reducing them down. Problem solved.

And, unfortunately, people who don’t download images, or can’t perceive images won’t get any alternate text, as there is no content <img> element any more. Those guys have always been last in the pecking order and there’s no reason to assume they’ll get a better deal this time.

And that’s why some agreement on <picture>, <pic>, <img srcset=”…”> (whatever it is; I don’t much care) needs to happen, and fast.

Update 22 June 2012: WHATWG conversation mysteriously dried, so Matthew Marquis wrote to W3C HTML WG and proposed Florian’s suggestion.

19 Responses to “ Why HTML5 urgently needs an HTML adaptive images mechanism ”

Comment by Matt Wilcox

Grrr!

Agreed, I’d like to see the <pic> compromise pushed now. We’re past the point of usefulness in further debate, we need to get it done and refine later.

/Grrr!

Comment by Charlie

Death to markup for behavioural purposes!

Sprinkle a little javascript on standard images to rewrite the source if the client wants higher *resolution* images for the same viewport size. Extend the bitmap formats to allow http servers to serve subimages by index but keep this out of the markup.

Otherwise you have to do the same dance for videos and it makes no sense at all for vector formats like SVG which can happily stick to the “old ways”. What? change the markup because I changed the image file format? Have you been at the meths again?!

Comment by Bruce

Charlie

“Sprinkle a little javascript on standard images to rewrite the source if the client wants higher *resolution* images for the same viewport size.”

.. and then download both images? How does that help? See “Pre-fetch; a spanner in the works” in http://dev.opera.com/articles/view/responsive-images-problem/

“Extend the bitmap formats to allow http servers to serve subimages” – because everything that runs HTML is connected to a server, right? (Think about epub, for example)

Comment by Charlie

Bruce,

swallowing a spider to catch the pre-fetch fly certainly isn’t the solution. And what do you do with video? Where you potentially have the same problem. The solution is not more markup. You might extend the http-headers or set some kind of viewport related flag.

Using a container image format, similar to .ico, works fine offline. Personally, I’d love to see Webp lead the way on this.

Comment by Anselm

Charlie,

I also would love to see WebP supporting different resolutions. This is what I already requested several times (http://anselm-hannemann.com/blog/2012/04/14/a-chance-for-webp/) but got no answer. Also it would take a long time to be able to actively use that (do you remember the lazyness of normal users? IE etc…).

But this does not mean by far that we do not need a responsive image solution within HTML. You can read why in Matt’s Opera Dev article or several others Bruce linked to.

And Bruce, I’d love to see a quick implementation of a syntax now. We are ready to do so but there’s the lack of activity in WHATWG and W3CG right now. Hope the push on W3 list of today helps :)

Comment by Remi Grumeau

Agree with Anselm, definitely something Google should had solve with their WebP to make it sexier.

Using the low-end image format by default and increase quality/size via Javascript sounds the best option to me for now, assuming a server-side script force the highest resolution for search engines… otherwise having all my website medias indexed in 240×120 by Google might be an issue.

I really don’t see a client-side only solution that could be compatible with low-end browsers. I also really think thinking about this problem as a screen size or DPI is totally missing the spot.

The real important thing with mobile web is bandwidth saving + making it quick to load. All the rest is marketing.
If my carrier signal sucks since i’m in the subway or -1 parking or highspeed train, i wouldn’t mind a low-res version on my Retina iPhone4 if i get it quick. In those situations, a crappy $30 droid or Symbian user would get to the point quicker than a $600 iPhone4S user? There’s no reason.

Also, i’m in Europe where good coverage 3G data plan incl 3Gb + unlimited calls costs €20 a month. For people having pay-as-you-go data plans in USA, that could make a hell of a sense to get a lighter (aka cheaper) version of images ;)
But for all of use, when laying on our home’s couch with a stable wifi connection, high-res retina is ok.

Type of connection, data plan & avail bandwidth. Those are the real questions we have to solve. Do some A/B testing for retina display users (between quick & low-res / slow & high-res) and see which solution please them the best.

Comment by Mr C

webkit-image-set isn’t perfect, but the problem appears to be nowhere near as bad as suggested here. adding a style tag to an IMG element such as the following, works just fine:

content:-webkit-image-set(…);

no need for background images and alt text still works. give this demo page a try if you’re using a retina mac + either chrome or safari: http://sidepodcast.com/playground/hidpi.html

Comment by Bruce

Mr C

the problem with using the CSS content property is that all browsers have an algorithm called prefetch in which they begin downloading images as soon as they are parsed in the HTML – before they even start to process JavaScript or CSS. This is done to make images appear faster (as they’re the heaviest elements on a page).

So by the time the CSS is parsed, the image is already partially (or even fully) downloaded, and only then potentially swapped out and a replacement downloaded.

Therefore, it potentially uses more bandwidth than just downloading the original higher-res-than necessary image. Opera Dragonfly/ Firebug/ Web Inspector / F12 tool shows this.

That’s why we need some way to add a mechanism to HTML, on the element itself, rather than rely on JavaScript or CSS.

Comment by Andy Davies

It’s just that “Webkit has implemented a new non-standard CSS function called -webkit-image-set:” won’t hold true for much longer.

TBH I’m not keen on either picture or srcset/image-set syntax as I think they both end up duplicating breakpoints multiple times – I wonder if there’s a better route with CSS and URI templates.

There has to be a better way but I think for better or worse we’re going to endup with what WHATWG want and the big weakness of WHATWG is it’s focused on the implementors not the developers (and they’re can be some unpleasantness because of that)

I think the WHATWG’s focus on implementors is why both image-set and setsrc have made it into the standard rather than something that’s perhaps more developer friendly.

I knew Apple had done the equivalent of a ‘drive by shooting’ on this but didn’t realise some of the undertones until I read your pastry box post on Sat morning. (unless I’m reading more into that post than there is)

Andy