jQuery accessible HTML5 video captioning plugin
Last month, I wrote an article on Accessible HTML5 Video with JavaScripted captions and dreamed that some clever JavaScripters would take it up and improve it, making it more generic so that a developer could mark up a transcript with timestamps, call in a library and -hey presto!- there would be captions.
Those nice chaps at 360innovate have started work on a jQuery plugin called jCaps.
As it’s all open licensed, if anyone fancies participating, it would be jolly. Here’s a wishlist that I’d love to see, and would write if I were not so scripting-challenged:
-
It would be useful to point to the transcriptions via the
aria-describedbyattribute on the video element, the values of which point to theidof an element containing the transcript (which could therefore be anarticle, adivor whatever). You can have multiple values onaria-describedby(like you can withclass) so that allows you to point to different translations.This allows you to have transcripts anywhere in the source, so greater flexibility for laying out the page.
-
If there are multiple transcripts for a single video, they need to have a
langattribute, and the script should construct aselectelement so the user can choose the language they wish to see transcripts in. (The first one offered should be the transcript that matches thelangattribute on thehtmlelement; if there isn’t one, the default should be the first in source order. -
I’d love to see a sexy (and stylable) skin around the
videoelement that had a YouTube-stylee user interface that houses all the buttons for turning on captions, transcripts and the language picker.
Anyone up for the challenge, and generous enough to release it as a BSD-licensed library/plug-in?
Note that this technique will have a limited shelf-life. The HTML Accessibility Working Group have two specifications that will enable captioning etc to be done natively once the spec is agreed and (of course) implemented.
See my related post: Some accessibility today or full accessibility jam tomorrow?
40 Responses to “ jQuery accessible HTML5 video captioning plugin ”
The dream of anyone being able to easily write captions to go with that Hitler Downfall YouTube video is so very close to fruition!
I have found an experiment here :
http://annodex.net/~silvia/itext/elephant_no_skin.html
(please excuse my English
)
[...] jCaps is nearing another release – hopefully this week. It will take into account some of Bruce Lawson’s wishlist, and includes a big refactoring of many elements of the plugin, including the API. It should be [...]
[...] sure how this is going to work”. But no one does. The closest thing I’ve found is a jQuery plugin that’s in the early [...]
When new technique comes out, will drop-in replacement be possible? (i.e. if user just uploads new version of the plugin, will it start using “jam” api?)
I think it would be best if it were future-proof, so that switch to new technique is hassle-free.
This comment was originally posted on Bruce Lawson’s personal site
Something that works right now in browsers is surely a far better option than waiting for something that might, or might not appear in the future. Future implementations are all vapourware until it’s implemented, and only useful when widely available.
This comment was originally posted on Bruce Lawson’s personal site
@kl
When the new APIs come on-stream, everyone will have to update their video elements anyway, to add new stuff for accessibility.
The temporary jCaps plugin takes a transcription on the page (not one pulled in as an .srt file within the extended video element) and makes captions from it.
So you’d need to remove the inclusion of the jQuery library (one line) as well as add all the new API stuff.
This comment was originally posted on Bruce Lawson’s personal site
Experimenting with “hacks” usually generates valuable feedback for what we should do for their native variant so I would urge people to keep on doing this!
This comment was originally posted on Bruce Lawson’s personal site
We used DOM Level 0 event handling for years until the newer event handling came along. And we support Microsoft’s way in addition to the DOM method, and the older DOM methods.
Future proof your technique. If the media API is stable enough, show how the media API can be incorporated safely today, even if it isn’t implemented (object detection). Web developers are used to these situations.
One thing you don’t necessarily want to do, is establish a pattern of, “Well, don’t do anything until we say, OK?”
This comment was originally posted on Bruce Lawson’s personal site
Is it possible to detect support for roles?
It would be really useful to be able to do something like:
if(@role.captions) {
# use API
}
else {
# use jCaps
}
This comment was originally posted on Bruce Lawson’s personal site
Well, you know me: I’m always in favor of a hack that mostly works today in favor of a total solution that might work tomorrow. As Anne says, you can get valuable insight into how tomorrow’s total solution should work by experimenting with today’s partial solution.
This comment was originally posted on Bruce Lawson’s personal site
More synchronicity Bruce, I blogged on *exactly* the same subject today! This is getting freaky! http://bit.ly/90G3GB
This comment was originally posted on Bruce Lawson’s personal site
Wow, John. Spookytastic! Synchronicity indeed. (reckon you code up the wishlist?)
Alastair, Shelly; there’s no future-proofing possible, really. My technique merely requires a transcript with timestamps to be associated with the video element via aria-describedby. The JS library does the rest.
When the new way is implemented cross-browser, you just need to remove the line that calls in the library. And add the stuff for the new API, of course. The transcript will still be there for other readers and SEO.
This comment was originally posted on Bruce Lawson’s personal site
Bruce, without having a chance to look at the Media API or jCaps, and going by past situations of this kind, what I’m assuming will happen is the jCaps people will move ahead with their work now, but continue the work, even as the new Media API releases.
If the library provides a level of abstraction, then those people who are using the library won’t have change their interface, because all of the changing will happen within the jCaps library. The jCaps people might expand the library, and add new capabilities, but the earlier functionality would still remain.
Regardless, you never tell people to stop. If you do, you’ll have the devil’s own time getting them going again.
This comment was originally posted on Bruce Lawson’s personal site
@Bruce, I’m happy to start bashing away at the wishlist. Most of it looks do-able, time allowing. I’m not happy with the way I originally wrote the API – it’s not jQuery-esque enough and I think there are some issues that will bite me on the bum further down the road.
I had also told your very good friend that I would pause development to get a full understanding of the issues, and a consensus over whether we should go ahead with this or not. It looks like it is something that will be useful, so I’m happy to proceed.
@Shelley:
“Regardless, you never tell people to stop. If you do, you’ll have the devil’s own time getting them going again.”
Ain’t that the truth!
This comment was originally posted on Bruce Lawson’s personal site
Actually there are several discussions at the WHATVG to integrate it in the standard. Silvia Pfeiffer of Mozilla is even making a proposal for accessible Video tag caption https://wiki.mozilla.org/Accessibility/HTML5_captions_v2
As HTML5 implementation goes fast we could see it soon in Firefox or Chrome.
This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer
Actually there are several discussions at the WHATVG to integrate it in the standard. Silvia Pfeiffer of Mozilla is even making a proposal for accessible Video tag caption https://wiki.mozilla.org/Accessibility/HTML5_captions_v2
As HTML5 implementation goes fast we could see it soon in Firefox or Chrome.
This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer
@sroucheray – While the implementation might come quickly for Firefox/Chrome – what about the rest of the browsers? Is it really a solution if it only hits less than 20% of the users on the internet? I assume that we would have to wait much later to this go through the entire standards process (probably 3-5 years) – and have it available as a part of the official spec. Probably in 5-7 years we could see it actually implemented in browsers for 75% of the users.
Flash provides it today for 98% of the Internet.
This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer
@sroucheray – While the implementation might come quickly for Firefox/Chrome – what about the rest of the browsers? Is it really a solution if it only hits less than 20% of the users on the internet? I assume that we would have to wait much later to this go through the entire standards process (probably 3-5 years) – and have it available as a part of the official spec. Probably in 5-7 years we could see it actually implemented in browsers for 75% of the users.
Flash provides it today for 98% of the Internet.
This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer
@David Tucker
Actually, Firefox alone has 24% of the world users. And, if I remember the chart correctly, Chrome comes in at around 9% – making the FF/Chrome bundle at 33%.
Still, though, I agree. Until the majority (that 60% held by IE) of world users can view this standard, without add-ons, HTML5 is not a viable method of delivering video content. Until then, it is only workable as a backup when Flash – or another 3rd party plug-in for that matter – is not available.
This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer
@David Tucker
Actually, Firefox alone has 24% of the world users. And, if I remember the chart correctly, Chrome comes in at around 9% – making the FF/Chrome bundle at 33%.
Still, though, I agree. Until the majority (that 60% held by IE) of world users can view this standard, without add-ons, HTML5 is not a viable method of delivering video content. Until then, it is only workable as a backup when Flash – or another 3rd party plug-in for that matter – is not available.
This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer
David – what you’re saying is right – but on the other hand, if you’re addressing the mobile web (1% of total traffic) then HTML 5 video using H264 as the codec is the pragmatic way to go.
The pragmatic solution I’d like to see would be just being able to embed .flv video within the tag, and then direct support for H264/.flv within browsers – as Adobe opens up the various Flash specs this should become possible.
The other thing is that we should never let today’s dominant technology prevent us heading in the right direction. Any time something new comes along (GUI systems, new programming languages, etc) there’s always someone to say ‘but most of the worlds code is in COBOL’, etc.
At one point no one used Flash for video – they used Real, Quicktime or WMP. It took some sites to take the lead and use it and prove it was the best available solution, rather than going with the safe option.
This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer
David – what you’re saying is right – but on the other hand, if you’re addressing the mobile web (1% of total traffic) then HTML 5 video using H264 as the codec is the pragmatic way to go.
The pragmatic solution I’d like to see would be just being able to embed .flv video within the tag, and then direct support for H264/.flv within browsers – as Adobe opens up the various Flash specs this should become possible.
The other thing is that we should never let today’s dominant technology prevent us heading in the right direction. Any time something new comes along (GUI systems, new programming languages, etc) there’s always someone to say ‘but most of the worlds code is in COBOL’, etc.
At one point no one used Flash for video – they used Real, Quicktime or WMP. It took some sites to take the lead and use it and prove it was the best available solution, rather than going with the safe option.
This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer
Flash for video was a safe solution for companies because 98% of users had the plugin installed. That’s the reason Flash video took off over the other video plugins which always had a smaller user base.
That said, today there’s certainly part of the audience you are not reaching with Flash video and it definitely makes sense to use the video tag as a fall back for mobile devices that support it.
Back to the main topic, people complain that Flash isn’t accessible yet there are a lot of big debates going on about things like the Canvas tag, which so far looks like it will be less accessible than Flash movie.
This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer
Flash for video was a safe solution for companies because 98% of users had the plugin installed. That’s the reason Flash video took off over the other video plugins which always had a smaller user base.
That said, today there’s certainly part of the audience you are not reaching with Flash video and it definitely makes sense to use the video tag as a fall back for mobile devices that support it.
Back to the main topic, people complain that Flash isn’t accessible yet there are a lot of big debates going on about things like the Canvas tag, which so far looks like it will be less accessible than Flash movie.
This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer
@Jules Why in the world HTML5 with h264 is the way to go in mobile? With 19/20 (go figure) mobile companies pursuing a way to add Flash Player to their browsers? Oh and did you see that Android video demos? That’s really neat!
This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer
@Jules Why in the world HTML5 with h264 is the way to go in mobile? With 19/20 (go figure) mobile companies pursuing a way to add Flash Player to their browsers? Oh and did you see that Android video demos? That’s really neat!
This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer
Understood. We’ve got to ride out the hype cycle though.
When campaigners “ride the tiger” by setting unrealistic expectations, they’ve got to manage the gradual deflation of that hype over time. I don’t think proponents are skilled at this, or even aware of this. Truth will out.
btw, there’s a ton of existing captioning creation & display tools available through websearch on “site:adobe.com captioning”. YouTube’s help will make captioning more of a baseline consumer expectation.
jd/adobe
This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer
Understood. We’ve got to ride out the hype cycle though.
When campaigners “ride the tiger” by setting unrealistic expectations, they’ve got to manage the gradual deflation of that hype over time. I don’t think proponents are skilled at this, or even aware of this. Truth will out.
btw, there’s a ton of existing captioning creation & display tools available through websearch on “site:adobe.com captioning”. YouTube’s help will make captioning more of a baseline consumer expectation.
jd/adobe
This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer
@sroucheray, that’s great. Another proposal on top of the tons of other implementations. I’m sure HTML5 will figure it out, but as Matthew points out, they’re just worried about the core functionality and not about implementing accessibility (yet).
Flash HAS done a lot of work to be more accessible and will probably be superior to HTML5 in terms of accessibility for a while.
@JulesLT, I like the idea of being able to use .flv as the source of a video tag. Not sure what it would take to do it, but it’s a cool idea.
=Ryan
ryan@adobe.com
This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer
@sroucheray, that’s great. Another proposal on top of the tons of other implementations. I’m sure HTML5 will figure it out, but as Matthew points out, they’re just worried about the core functionality and not about implementing accessibility (yet).
Flash HAS done a lot of work to be more accessible and will probably be superior to HTML5 in terms of accessibility for a while.
@JulesLT, I like the idea of being able to use .flv as the source of a video tag. Not sure what it would take to do it, but it’s a cool idea.
=Ryan
ryan@adobe.com
This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer
Actually there are several discussions at the WHATVG to integrate it in the standard. Silvia Pfeiffer of Mozilla is even making a proposal for accessible Video tag caption https://wiki.mozilla.org/Accessibility/HTML5_captions_v2
As HTML5 implementation goes fast we could see it soon in Firefox or Chrome.
This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer
@sroucheray – While the implementation might come quickly for Firefox/Chrome – what about the rest of the browsers? Is it really a solution if it only hits less than 20% of the users on the internet? I assume that we would have to wait much later to this go through the entire standards process (probably 3-5 years) – and have it available as a part of the official spec. Probably in 5-7 years we could see it actually implemented in browsers for 75% of the users.
Flash provides it today for 98% of the Internet.
This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer
Flash for video was a safe solution for companies because 98% of users had the plugin installed. That’s the reason Flash video took off over the other video plugins which always had a smaller user base.
That said, today there’s certainly part of the audience you are not reaching with Flash video and it definitely makes sense to use the video tag as a fall back for mobile devices that support it.
Back to the main topic, people complain that Flash isn’t accessible yet there are a lot of big debates going on about things like the Canvas tag, which so far looks like it will be less accessible than Flash movie.
This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer
@Jules Why in the world HTML5 with h264 is the way to go in mobile? With 19/20 (go figure) mobile companies pursuing a way to add Flash Player to their browsers? Oh and did you see that Android video demos? That’s really neat!
This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer
Understood. We’ve got to ride out the hype cycle though.
When campaigners “ride the tiger” by setting unrealistic expectations, they’ve got to manage the gradual deflation of that hype over time. I don’t think proponents are skilled at this, or even aware of this. Truth will out.
btw, there’s a ton of existing captioning creation & display tools available through websearch on “site:adobe.com captioning”. YouTube’s help will make captioning more of a baseline consumer expectation.
jd/adobe
This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer
I’m a Flash and JavaScript developer and I like Flash and AS3 as much as JavaScript. I must mention that Paul Rouget of Mozilla made a caption demo a year ago : http://blog.mozbox.org/post/2009/03/10/video-tag-and-subtitles
Sure, HTML5 won’t kill Flash soon but you guys at Adobe when you are as much on defense, you let me think that… maybe, you’re afraid of HTML5 and thus you give it great value.
This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer
My point isn’t that it’s not possible, it definitely is. My point is that there’s no standard for captions in the HTML5 spec. That means you’re going to end up with 50,000 different implementations of it with varying support for formats and varying levels of quality.
That’s not a bad thing in and of itself, but it makes it harder for YouTube to flip the switch on captions for HTML5 video.
Not a problem they have with Flash.
=Ryan
ryan@adobe.com
This comment was originally posted on Ryan Stewart – Rich Internet Application Mountaineer
“Those nice chaps at 360innovate have started work on a jQuery plugin called jCaps.”
Wait… I thought that plugins were eevil…?
jd/adobe