Bruce Lawson’s personal site

jQuery accessible HTML5 video captioning plugin

Last month, I wrote an article on Accessible HTML5 Video with JavaScripted captions and dreamed that some clever JavaScripters would take it up and improve it, making it more generic so that a developer could mark up a transcript with timestamps, call in a library and -hey presto!- there would be captions.

Those nice chaps at 360innovate have started work on a jQuery plugin called jCaps.

As it’s all open licensed, if anyone fancies participating, it would be jolly. Here’s a wishlist that I’d love to see, and would write if I were not so scripting-challenged:

  1. It would be useful to point to the transcriptions via the aria-describedby attribute on the video element, the values of which point to the id of an element containing the transcript (which could therefore be an article, a div or whatever). You can have multiple values on aria-describedby (like you can with class) so that allows you to point to different translations.

    This allows you to have transcripts anywhere in the source, so greater flexibility for laying out the page.

  2. If there are multiple transcripts for a single video, they need to have a lang attribute, and the script should construct a select element so the user can choose the language they wish to see transcripts in. (The first one offered should be the transcript that matches the lang attribute on the html element; if there isn’t one, the default should be the first in source order.

  3. I’d love to see a sexy (and stylable) skin around the video element that had a YouTube-stylee user interface that houses all the buttons for turning on captions, transcripts and the language picker.

Anyone up for the challenge, and generous enough to release it as a BSD-licensed library/plug-in?

Note that this technique will have a limited shelf-life. The HTML Accessibility Working Group have two specifications that will enable captioning etc to be done natively once the spec is agreed and (of course) implemented.

See my related post: Some accessibility today or full accessibility jam tomorrow?

40 Responses to “ jQuery accessible HTML5 video captioning plugin ”

Comment by kl

When new technique comes out, will drop-in replacement be possible? (i.e. if user just uploads new version of the plugin, will it start using “jam” api?)

I think it would be best if it were future-proof, so that switch to new technique is hassle-free.

This comment was originally posted on Bruce Lawson’s personal site

Comment by Isofarro

Something that works right now in browsers is surely a far better option than waiting for something that might, or might not appear in the future. Future implementations are all vapourware until it’s implemented, and only useful when widely available.

This comment was originally posted on Bruce Lawson’s personal site

Comment by Bruce

@kl

When the new APIs come on-stream, everyone will have to update their video elements anyway, to add new stuff for accessibility.

The temporary jCaps plugin takes a transcription on the page (not one pulled in as an .srt file within the extended video element) and makes captions from it.

So you’d need to remove the inclusion of the jQuery library (one line) as well as add all the new API stuff.

This comment was originally posted on Bruce Lawson’s personal site

Comment by Shelley

We used DOM Level 0 event handling for years until the newer event handling came along. And we support Microsoft’s way in addition to the DOM method, and the older DOM methods.

Future proof your technique. If the media API is stable enough, show how the media API can be incorporated safely today, even if it isn’t implemented (object detection). Web developers are used to these situations.

One thing you don’t necessarily want to do, is establish a pattern of, “Well, don’t do anything until we say, OK?”

This comment was originally posted on Bruce Lawson’s personal site

Comment by Eric Meyer

Well, you know me: I’m always in favor of a hack that mostly works today in favor of a total solution that might work tomorrow. As Anne says, you can get valuable insight into how tomorrow’s total solution should work by experimenting with today’s partial solution.

This comment was originally posted on Bruce Lawson’s personal site

Comment by Bruce

Wow, John. Spookytastic! Synchronicity indeed. (reckon you code up the wishlist?)

Alastair, Shelly; there’s no future-proofing possible, really. My technique merely requires a transcript with timestamps to be associated with the video element via aria-describedby. The JS library does the rest.

When the new way is implemented cross-browser, you just need to remove the line that calls in the library. And add the stuff for the new API, of course. The transcript will still be there for other readers and SEO.

This comment was originally posted on Bruce Lawson’s personal site

Comment by Shelley

Bruce, without having a chance to look at the Media API or jCaps, and going by past situations of this kind, what I’m assuming will happen is the jCaps people will move ahead with their work now, but continue the work, even as the new Media API releases.

If the library provides a level of abstraction, then those people who are using the library won’t have change their interface, because all of the changing will happen within the jCaps library. The jCaps people might expand the library, and add new capabilities, but the earlier functionality would still remain.

Regardless, you never tell people to stop. If you do, you’ll have the devil’s own time getting them going again.

This comment was originally posted on Bruce Lawson’s personal site

Comment by John McCollum

@Bruce, I’m happy to start bashing away at the wishlist. Most of it looks do-able, time allowing. I’m not happy with the way I originally wrote the API – it’s not jQuery-esque enough and I think there are some issues that will bite me on the bum further down the road.

I had also told your very good friend that I would pause development to get a full understanding of the issues, and a consensus over whether we should go ahead with this or not. It looks like it is something that will be useful, so I’m happy to proceed.

@Shelley:
“Regardless, you never tell people to stop. If you do, you’ll have the devil’s own time getting them going again.”

Ain’t that the truth!

This comment was originally posted on Bruce Lawson’s personal site

Comment by David Tucker

@sroucheray – While the implementation might come quickly for Firefox/Chrome – what about the rest of the browsers? Is it really a solution if it only hits less than 20% of the users on the internet? I assume that we would have to wait much later to this go through the entire standards process (probably 3-5 years) – and have it available as a part of the official spec. Probably in 5-7 years we could see it actually implemented in browsers for 75% of the users.

Flash provides it today for 98% of the Internet.

This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer

Comment by David Tucker

@sroucheray – While the implementation might come quickly for Firefox/Chrome – what about the rest of the browsers? Is it really a solution if it only hits less than 20% of the users on the internet? I assume that we would have to wait much later to this go through the entire standards process (probably 3-5 years) – and have it available as a part of the official spec. Probably in 5-7 years we could see it actually implemented in browsers for 75% of the users.

Flash provides it today for 98% of the Internet.

This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer

Comment by Ross R

@David Tucker
Actually, Firefox alone has 24% of the world users. And, if I remember the chart correctly, Chrome comes in at around 9% – making the FF/Chrome bundle at 33%.

Still, though, I agree. Until the majority (that 60% held by IE) of world users can view this standard, without add-ons, HTML5 is not a viable method of delivering video content. Until then, it is only workable as a backup when Flash – or another 3rd party plug-in for that matter – is not available.

This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer

Comment by Ross R

@David Tucker
Actually, Firefox alone has 24% of the world users. And, if I remember the chart correctly, Chrome comes in at around 9% – making the FF/Chrome bundle at 33%.

Still, though, I agree. Until the majority (that 60% held by IE) of world users can view this standard, without add-ons, HTML5 is not a viable method of delivering video content. Until then, it is only workable as a backup when Flash – or another 3rd party plug-in for that matter – is not available.

This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer

Comment by JulesLt

David – what you’re saying is right – but on the other hand, if you’re addressing the mobile web (1% of total traffic) then HTML 5 video using H264 as the codec is the pragmatic way to go.

The pragmatic solution I’d like to see would be just being able to embed .flv video within the tag, and then direct support for H264/.flv within browsers – as Adobe opens up the various Flash specs this should become possible.

The other thing is that we should never let today’s dominant technology prevent us heading in the right direction. Any time something new comes along (GUI systems, new programming languages, etc) there’s always someone to say ‘but most of the worlds code is in COBOL’, etc.

At one point no one used Flash for video – they used Real, Quicktime or WMP. It took some sites to take the lead and use it and prove it was the best available solution, rather than going with the safe option.

This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer

Comment by JulesLt

David – what you’re saying is right – but on the other hand, if you’re addressing the mobile web (1% of total traffic) then HTML 5 video using H264 as the codec is the pragmatic way to go.

The pragmatic solution I’d like to see would be just being able to embed .flv video within the tag, and then direct support for H264/.flv within browsers – as Adobe opens up the various Flash specs this should become possible.

The other thing is that we should never let today’s dominant technology prevent us heading in the right direction. Any time something new comes along (GUI systems, new programming languages, etc) there’s always someone to say ‘but most of the worlds code is in COBOL’, etc.

At one point no one used Flash for video – they used Real, Quicktime or WMP. It took some sites to take the lead and use it and prove it was the best available solution, rather than going with the safe option.

This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer

Comment by Matthew Fabb

Flash for video was a safe solution for companies because 98% of users had the plugin installed. That’s the reason Flash video took off over the other video plugins which always had a smaller user base.

That said, today there’s certainly part of the audience you are not reaching with Flash video and it definitely makes sense to use the video tag as a fall back for mobile devices that support it.

Back to the main topic, people complain that Flash isn’t accessible yet there are a lot of big debates going on about things like the Canvas tag, which so far looks like it will be less accessible than Flash movie.

This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer

Comment by Matthew Fabb

Flash for video was a safe solution for companies because 98% of users had the plugin installed. That’s the reason Flash video took off over the other video plugins which always had a smaller user base.

That said, today there’s certainly part of the audience you are not reaching with Flash video and it definitely makes sense to use the video tag as a fall back for mobile devices that support it.

Back to the main topic, people complain that Flash isn’t accessible yet there are a lot of big debates going on about things like the Canvas tag, which so far looks like it will be less accessible than Flash movie.

This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer

Comment by John Dowdell

Understood. We’ve got to ride out the hype cycle though.

When campaigners “ride the tiger” by setting unrealistic expectations, they’ve got to manage the gradual deflation of that hype over time. I don’t think proponents are skilled at this, or even aware of this. Truth will out.

btw, there’s a ton of existing captioning creation & display tools available through websearch on “site:adobe.com captioning”. YouTube’s help will make captioning more of a baseline consumer expectation.

jd/adobe

This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer

Comment by John Dowdell

Understood. We’ve got to ride out the hype cycle though.

When campaigners “ride the tiger” by setting unrealistic expectations, they’ve got to manage the gradual deflation of that hype over time. I don’t think proponents are skilled at this, or even aware of this. Truth will out.

btw, there’s a ton of existing captioning creation & display tools available through websearch on “site:adobe.com captioning”. YouTube’s help will make captioning more of a baseline consumer expectation.

jd/adobe

This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer

Comment by ryanstewart

@sroucheray, that’s great. Another proposal on top of the tons of other implementations. I’m sure HTML5 will figure it out, but as Matthew points out, they’re just worried about the core functionality and not about implementing accessibility (yet).

Flash HAS done a lot of work to be more accessible and will probably be superior to HTML5 in terms of accessibility for a while.

@JulesLT, I like the idea of being able to use .flv as the source of a video tag. Not sure what it would take to do it, but it’s a cool idea.

=Ryan
ryan@adobe.com

This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer

Comment by ryanstewart

@sroucheray, that’s great. Another proposal on top of the tons of other implementations. I’m sure HTML5 will figure it out, but as Matthew points out, they’re just worried about the core functionality and not about implementing accessibility (yet).

Flash HAS done a lot of work to be more accessible and will probably be superior to HTML5 in terms of accessibility for a while.

@JulesLT, I like the idea of being able to use .flv as the source of a video tag. Not sure what it would take to do it, but it’s a cool idea.

=Ryan
ryan@adobe.com

This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer

Comment by David Tucker

@sroucheray – While the implementation might come quickly for Firefox/Chrome – what about the rest of the browsers? Is it really a solution if it only hits less than 20% of the users on the internet? I assume that we would have to wait much later to this go through the entire standards process (probably 3-5 years) – and have it available as a part of the official spec. Probably in 5-7 years we could see it actually implemented in browsers for 75% of the users.

Flash provides it today for 98% of the Internet.

This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer

Comment by Matthew Fabb

Flash for video was a safe solution for companies because 98% of users had the plugin installed. That’s the reason Flash video took off over the other video plugins which always had a smaller user base.

That said, today there’s certainly part of the audience you are not reaching with Flash video and it definitely makes sense to use the video tag as a fall back for mobile devices that support it.

Back to the main topic, people complain that Flash isn’t accessible yet there are a lot of big debates going on about things like the Canvas tag, which so far looks like it will be less accessible than Flash movie.

This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer

Comment by John Dowdell

Understood. We’ve got to ride out the hype cycle though.

When campaigners “ride the tiger” by setting unrealistic expectations, they’ve got to manage the gradual deflation of that hype over time. I don’t think proponents are skilled at this, or even aware of this. Truth will out.

btw, there’s a ton of existing captioning creation & display tools available through websearch on “site:adobe.com captioning”. YouTube’s help will make captioning more of a baseline consumer expectation.

jd/adobe

This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer

Comment by sroucheray

I’m a Flash and JavaScript developer and I like Flash and AS3 as much as JavaScript. I must mention that Paul Rouget of Mozilla made a caption demo a year ago : http://blog.mozbox.org/post/2009/03/10/video-tag-and-subtitles
Sure, HTML5 won’t kill Flash soon but you guys at Adobe when you are as much on defense, you let me think that… maybe, you’re afraid of HTML5 and thus you give it great value.

This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer

Comment by ryanstewart

My point isn’t that it’s not possible, it definitely is. My point is that there’s no standard for captions in the HTML5 spec. That means you’re going to end up with 50,000 different implementations of it with varying support for formats and varying levels of quality.

That’s not a bad thing in and of itself, but it makes it harder for YouTube to flip the switch on captions for HTML5 video.

Not a problem they have with Flash.

=Ryan
ryan@adobe.com

This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer

Leave a Reply

HTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> . To display code, manually escape it.