I use ffmpeg to add some information overlays to the videos that I often upload to YouTube. I’ve documented these before, but I had a problem that arose from time to time that I never figured out: occasionally my encodes would seemingly just go on forever, and never terminate. I discovered today that this was because the use of the “overlay” filter. It apparently keeps generating frames even after the main stream has terminated, which seems odd to me, but… whatever. If you add the repeatlast argument, you’ll get what you want.
Anyone who has seen my projects on the Atari 2600 might reasonably conclude that I have a thing for retro computing. The saying goes “it is no virtue to do with more, what can be done with less” and I can’t think of someone whose projects have embodied that more than demo coder Linus Akesson (lft). His latest demo runs on the tiniest of tiny Atmel processors, the ATTINY15. It has just 32 bytes of memory (just the registers, really) and room for 512 instructions. Yet, lft made a cool demo that runs on it, generating both video and sound. Oh, and he’s only running it at 1.6 Mhz (yes, 1.6 Megahertz). Very cool!
One year ago today, I first published a link to The Broadcaster Project, a site which had several tips on using command line tools such as ffmpeg to assemble videos. I use a similar technique to do my more recent videos: I take the raw footage from the camera and resize it, denoise it, and add some overlaid text before uploading it to YouTube. Revisiting The Broadcaster Project, I see they have a few more recipes that can be useful, such as ones for assembling time lapse movies and the like. I also found a reference to youtube-upload, a command line python script that can automatically upload videos from the command line, a part of my process that I still did with the web interface.
I’ve received two requests for information about my “video production pipeline”, such as it is. As you can tell by my videos, I am shooting with pretty ugly hardware, in a pretty ugly way, with minimal (read “no”) editing. But I did figure out a pretty nice way to add some watermarks and overlays to my videos using open source tools like ffmpeg, and thought that it might be worth documenting here (at least so I don’t forget how to do it myself).
First of all, I’m using an old Kodak Zi6 that I got for a good price off woot.com. It shoots at 1280x720p, which is a nominally a widescreen HD format. But since ultimately I am going to target YouTube, and because the video quality isn’t all that astounding anyway, I have chosen in all my most recent videos to target a 480 line format, which (assuming 16:9) aspect ratio means that I need tor resize my videos down to 854×480. The Zi6 saves in a Quicktime container format, using h264 video and AAC audio at 8Mbps and 128kbps respectively.
For general mucking around with video, I like to use my favorite Swiss army knife: ffmpeg. It reads and writes a ton of formats, and has a nifty set of features that help in titling. You can try installing it from whatever binary repository you like, but I usually find out that I need to rebuild it to include some option that the makers of the binary repository didn’t think to add. Luckily, it’s not really that hard to build: you can follow the instructions to get a current source tree, and then it’s simply a matter of building it with the needed options. If you run ffmpeg by itself, it will tell you what the configuration options it used for compilation. For my own compile, I used these options:
I enabled libvpx, libvorbis and libtheora for experimenting with webm and related codecs. I added libx264 and libfaac so I could do MPEG4, mp3lame so I could encode to mp3 format audio, most important for this example, libfreetype so it would build video filters that could overlay text onto my video. If you compile ffmpeg with these options, you should be compatible with what I am doing here.
It wouldn’t be hard to just type a quick command line to resize and re-encode the video, but I’m after something a bit more complicated here. My pipeline resizes, removes noise, does a fade in, and then adds text over the bottom of the screen that contains the video title, my contact information, and a Creative Commons banner so that people know how they can reuse the video. To do this, I need to make use of the libavfilter features of ffmpeg. Without further ado, here’s the command line I used for a recent video:
So, what does all this do? Well, walking through the command: -y says to go ahead and overwrite the output file. -i specifies the input file to be the raw footage that I transferred from my Zi6. I specified -sameq to keep the quality level the same for the output:: you might want to specify an audio and video bitrate separately here, but I figure retaining the original quality for upload to YouTube is a good thing. I am shooting 16:9, so I specify the aspect ratio with the next argument.
Then comes the real magic: the -vf argument specifies a rather long command string. You can think of it as a series of chains, separated by semicolons. Each command in the chain is separated by commas. Inputs and outputs are specified by names appearing inside square brackets. Read the rather terse and difficult documentation if you want to understand more, but it’s not too hard to walk through what the chains do. From ‘scale” to the first semicolon, the input video (implicit input to the filter chain) we resize the video to the desired output size, fade in from black over the first 30 frames, and then run the high quality 3d denoiser, storing the result in register xxx. The next command creates a semi-transparent background color card which is 64 pixels high and the full width of the video, storing it in y. The next command takes the resized video xxx, and the color card yyy, and overlays the color at the bottom. We could store that in a new register, but instead we simply chain on a drawtext command. It specifies an x, y, and fontfile, as well as a file “hdr” which contains the text that we want to insert. For this video, that file looks like:
The (too simple) Micro FM transmitter on a breadboard
https://brainwagon.org | @brainwagon | mailto:brainwagon@gmail.com
The command then stores the output back in the register [xxx]. The next command reads a simple png file of the Creative Commons license and makes it available as a movie in register yyy. In this case, it’s just a simple .png, but you could use an animated file if you’d rather. The last command then takes xxx and yyy and overlays them so that the copyright appears in the right place.
And that’s it! To process my video, I just download from the camera to my linux box, change the title information in the file “hdr”, and then run the command. When it is done, I’m ready to upload the file to YouTube.
A couple of improvements I have yet to add to my usual pipeline: I don’t really like the edge of the transparent color block: it would be nicer to use a gradient. I couldn’t figure out how to synthesize one in ffmpeg, but it isn’t hard if you have something like the netpbm utilities:
Running this command builds a file called ‘mix.png’, which you can read in using the movie node in the avfilter chain, just as I did for the Creative Commons logo. Here’s an example:
If I were a real genius, I’d merge this whole thing into a little python script that could also manage the uploading to YouTube.
If this is any help to you, or you have refinements to the basic idea that you’d like to share, be sure to add your comments or links below!
I think I heard mention of this project during a recent episode of the FLOSS podcast a couple of weeks ago, and thought that it was an interesting idea, so remembered to track it down this morning. The basic idea is to provide a “loopback” device for video. You could write a program which can send out video frames, which are then read by any device that can read from v4l2 devices (like mplayer or mencoder). I have a couple of ideas for how this could be fun.
Sadly, typing “make” didn’t have the desired effect on my “Jaunty Jackalope” machine, and looking back, it seems that it hasn’t had an update since March 2008. It’s probably a fairly minor thing. I’ll try to work out the details later, and see if I can get it going.
Addendum: Some more playing around yielded some new links of interest. The v4l2vd project mentioned the older vloopback, which is mostly unsupported, but which is apparently tagging along with the motion project, which does software motion detection on webcams (and notably v4l devices). You can find the software and such here, including a test application which accepts video from one device, and writes it back out to another.
I got onto this stuff by looking at the flashcam project, which I found while trying to find something like ustream that could run on Linux.
Got this from digg, this entire short film was created by using the burst mode of the Canon 20D. The shots look really nice, with lots of great depth of field and exposure. Previously, I blogged about how The Corpse Bride was filmed using digital still photography, but this is the first time I think I’ve seen this particular trick, and it works really well. Worth checking out.
Incidently, a quick check at dpreview.com reveals that the Canon 20D burst mode is 5 fps until the buffer fills, then it slows to about 1.4 fps. I suspect that at smaller resolutions, much longer times at the full fps are maintained. Neat.
One of the somewhat interesting features of Google Video is now that you can put video from Google into your webpage. Here’s a Snowman video from WGBH Boston, and below you can see the player embedded in a web posting.
Deleted the video player
Something about WordPress insists on rewriting the contents of my post and breaks the formatting. I’ll figure it out later. If you figure it out, let let me know.