Playing with YouTube Captions

I’ve been playing this week with closed captions on YouTube. YouTube announced support for closed captions in August 2008, and followed that announcement with this demonstration video. This inspired us at DO-IT to create our own YouTube Channel and to start uploading captioned videos.

YouTube supports captions in either of two formats, SubViewer (.sub) or SubRip (.srt). The formats are described, with examples, on the YouTube Help page
Getting Started: Adding / Editing captions. Until recently I hadn’t fully explored YouTube’s caption support in any great detail, but here’s what I discovered this week:

According to the closest thing I could find to a SubViewer Specification, the SubViewer format includes an optional heading section, which allows authors to control the font family, size, and color of the caption text. These stylistic properties are supported by some media players, but not the YouTube player. I’m actually ok with that, as I think the appearance of captions should be controlled by the user, and commendably YouTube has taken some positive steps in that direction:

• To increase text size: press “+” key
• TO decrease text size: press “-” key
• To change background: press “B” or “b” key

Note that the YouTube player must first have focus before any of these keys work. Therefore, it requires a mouse click (or, in Internet Explorer, the user can tab into the player).

I had similarly little success finding a formal specification for SubRip (the other format supported by YouTube), but according to McPoodle, this format has no head section where universal font styles can be specified, but does support presentational markup within each caption, specifically… um… <font>, <i>, and <b>. YouTube doesn’t support this though, and again: I’m ok with that.

Since YouTube gives users control over font size, I find myself questioning whether we should force line breaks in our captions. For example, if a caption includes two short sentences, with a line break at the end of the first, that may be very readable in the default font size, but in a larger font the line break may force the captions to expand onto three lines, with one lonely word on line 2. For the DO-IT videos, we’ve decided to use line breaks for videos where the user has no control over caption size (e.g., Windows Media and Quicktime), but we remove those line breaks in caption files that will be delivered in players where users do have control, such as the YouTube player.

Keep Your Captions Short

The maximum number of lines that the YouTube caption player will display seems to be three. There also seems to be a limit to the number of characters, as seen in the following http://onhealthy.net/product-category/gastrointestinal-tract/ example. The example shows a caption in which the word “caption” should appear at the end of line 3, and would easily fit within the width of the video player, but it has been omitted, presumably due to a character limit.

The extra word doesn’t show up in a subsequent caption either. In fact, if we added an entire fourth line (I tried this), that line would never show up. However, there seems to be a threshold after which YouTube decides a caption is not just long, but really really long. That’s when the player kicks in to a special mode that I call really really long mode. It splits the caption into multiple captions, then displays each caption for intervals of equal length until the next time code from the original caption file rolls around. This is a pretty smart feature, but the fact that it doesn’t just do this automatically is a bug. Below is a screen shot of the test video in which I tried to figure out what the threshold was. In the caption on the screen, the word "really" appears six times. There is additional caption text that overflows and is never displayed. I had to add "really" a total of eighteen times before YouTube finally kicked into really really long mode and intelligently displayed the overflow.

Don’t Try This at Home

YouTube’s really really long mode might have some of you thinking you could just upload a transcript and let YouTube automatically divide it into captions. This, in fact, is true! The only time code that’s required is the start time (yes, I tried it). As described in the preceding section, YouTube breaks the transcript into captions and distributes them evenly over the length of the video. This is the same idea that’s behind UW-Madison’s World Caption with one big difference: after divvying up the captions, World Caption provides an interface that allows users to adjust the captions when they get out of sync. Without making adjustments, the captions and video won’t stay synchronized for long and the user who depends on captions is left trying to understand a confusing mess. So, until YouTube can convert your transcript to captions on the fly (using speech processing technology), you’ll need to create the captions yourself. There’s no reason not to, especially if you already have a transcript since getting the transcript is the hard part. There are plenty of tools and services for converting transcripts to captions, several of which are listed on YouTube’s Help With Captions page.