Category Archives: Flash

FLV to SWF and back

During the past couple of weeks, I’ve been working on a library that can convert SWF files to FLV files and vice versa. I’ve finally succeeded and release v0.10 tonight! If you are a media expert (or amateur!) I’d love some help testing this. Currently, I’ve only tested on Mac with FLV and SWF files generated by the On2 Flix Exporter for QuickTime. Media from other sources and testing on other platforms would be greatly appreciated.

The SWF and FLV file formats are media containers invented by Macromedia (now Adobe) for playback in Flash. Each file format has its own purpose, but with regard to just video and audio they are quite similar under the hood:

Both are tag-oriented container formats. FLV supports three types of tags (video, audio and meta) while SWF supports over 60 (video, audio, bytecode, shapes, sprites, fonts, etc) and is still growing. FLV represents a static media stream, like many of the QuickTime and Windows Media audio/video formats. SWF, on the other hand, is animated vector art with code, like SVG. Macromedia has been quite good about documenting the file formats, although they typically delay release of documentation for a generation (i.e. the Flash 6 documentation was published when Flash 7 came out) and require a license agreement.

The two tags types that SWF and FLV have in common (video and audio) share the same binary format for the media encoding, like MP3, H.263, VP6, ADCRM, ScreenVideo, NellyMoser, etc. Consequently it is feasible to convert audio and video data between the two file formats without requiring time-consuming transcoding or patented media encoding algorithms.

There are several complications involved in the conversion, however, that I had to overcome. For example SWF insists that there may be at most one audio packet per video frame and infers time from the video framerate. FLV on the other hand uses absolute time codes for every packet, whether video or audio.

While I was at it, I threw in an FLV to MP3 converter too since that’s just a subset of the FLV to SWF converter, and was handy for debugging.

I certainly have to give credit to the Ruby FLVTool2 library for helping me get v0.01 of my library (the initial FLV parsing part) out the door. That library helped me discover one place where Macromedia’s documentation was erroneous and helped me understand the undocumented FLV meta tag. Additionally, I must credit the SWF::File Perl library, which handles the SWF read/write portions of my transcoder. Yay open source!

The FlipBook user interface

The CoverFlow application is a technology demo that explores a clever way to browse your music collection. It attempts to loosely emulate the experience of flipping through a pile of albums or CDs. The application displays the album art for every album in your iTunes library sorted by artist or album title, and lets you click or scroll through that list in a glassy, 3D-looking showcase.

Most everyone is a fan of clever user interfaces, right? Even more so, we’re fans of clever UIs that are intuitive and efficient. CoverFlow achieves that to a remarkable degree. It superficially resembles Apple’s Front Row application, with it’s 3D icons on a glassy surface within an infinite black space. CoverFlow appears to predate Front Row, however, and in my opinion blows away Front Row for usability.

Interested? Well, CoverFlow is for Mac OS X 10.4 only and requires a modern video card. If that’s not you and you want to get a taste, take a look at my Flash implementation. I’ve recreated the basics of CoverFlow’s look in this functional prototype. This experiment is not specific to music, but instead is a general purpose navigation strategy. I’ve tentatively called this user interface a “FlipBook”. I’m not fond of that name, but I haven’t come up with one I like better yet…

Screenshot of FlipBook UI

Screen shot of the Flash implementation in action

My Flash version loads a list of items from an XML file. For each item in the list, the XML file specifies an image (JPG or SWF only), a label and an optional URL that is invoked on a double-click. The label is rendered in Flash’s subset of HTML and can accept one or more CSS files to style the text.

Currently (as of version 0.02), the user interaction is limited to clicking on items to center them and double-clicking on them to launch their associated URLs.

Is there a future in this? Maybe. Coverflow is very popular, and I can easily imagine the author making a nice bit of cash selling the implementation to Apple for inclusion in Front Row itself. As a general-purpose nagivation tool (i.e. not specific to music), this is a little harder since the one-dimensional nature of the flow of art is limiting. Nonetheless, this UI is more browseable than most online product catalogs. One could imagine a bookshelf implemented like this, showing book covers instead of album covers. Or perhaps a collection of videos of university lectures, organized by department and class.

When caches go bad

In general, caching is a good thing. Creating a temporary local copy of some data can dramatically speed up repeated access to that data, especially if it’s hard to recreate or slow to access. However, if the local copy falls out of sync with the real data it can sometimes be disastrous. Most of the hard work in caching is finding ways to ensure that the cached copy is fresh. This is called ensuring “cache coherency”.

For example, web browsers ask web servers if the remote content is newer than some time. If yes, then the cache is deleted and a new copy is fetched. If not, then the cache is used. This saves time because the simple “Is it fresh?” question is only a few bytes instead of the full size of the content. IE 5 had a painful misfeature that it didn’t send the “Is it fresh?” question for subsidiary files, like .css, .js, .jpg, etc files. So, even if the main .html page was up to date, sometimes IE used the wrong version of the supporting data.

Flash caches incorrectly

Last week I discovered a collosal caching problem with Flash MX 2004 (the authoring software, not the Flash Player that is in plugged into your browser). When switching to MX 2004, most users (me included) notice that compile time is dramatically improved. One of the many reasons is that Flash caches object files behind the scenes. That is, if you have a class defined in a .as file, Flash compiles it to a .aso file and stores that object file deep in a prefs directory. On subsequent compilations, if the .as file is older than the .aso file, the latter is reused, possibly saving several seconds.

Well, Macromedia screwed it up. By making the test be “Is it older?” they opened themselves up to a cache coherency problem. Consider this scenario:

Say, Alice and Bob are working on the same CVS project. Alice checks out some code one morning and starts programming. She makes some changes to Foo.as. Meanwhile, Bob speedily makes a critical bug fix to Bar.as and commits his change. An hour later, Alice finishes her Foo.as work and compiles her .fla. It fails because of the bug in Bar.as. Bob tells her, “I fixed that bug. Just do a cvs update and recompile.” She does, but the bug persists.

Why? It’s all about timestamps. Here’s what happened:

  • Alice checked out Bar.as with timestamp of 10:00 AM
  • Bob edited and checked in Bar.as with timestamp of 10:30 AM
  • Alice compiled at 11:00 AM, creating a Bar.aso file with a 11:00 AM timestamp
  • Alice cvs updated at 11:30 AM, getting Bar.as with a 10:30 AM timestamp
  • Alice recompiled her .fla, but Bar.aso is “newer” than Bar.as, so Flash ignored Bar.as and used the buggy Bar.aso instead.

So, Alice is stuck with an out-of-date .aso file that Flash won’t fix.

Workarounds

There are two easy workarounds to this problem, but both of them require manual intervention.

  1. Touch the .as file

    By editing the .as files, you make it newer again and Flash rebuilds the .aso. Instead of manually editing, the Unix “touch” command comes in handy. Here’s a one-line command to touch all of your project .as files to ensure they are newest:

    find . -name '*.as' | xargs touch
    

    If you do a lot of Actionscript work, though, this is a weak workaround becuase you may have updated .as files somewhere else in your classpath.

  2. Blow away the .aso files

    Like in the browser world, the answer here is “Clear your cache!” By deleting the .aso files you can force Flash to rebuild them all, at the expense of compile time. The trick is that you need to know where they are. Here’s a command that works on Mac OS X only (all one line):

    find ~/Library/Application*Support/Macromedia/Flash*MX*2004/en/Conf*/Classes \
        -name '*.aso' | perl -lpe 'unlink $_'
    

You have to remember to run one of these every time you have a case where you might introduce an older-but-changed .as file (often from a source repository like CVS or from a network drive).

What Macromedia should do

How can Macromedia fix this problem? Very simply! This problem was solved by caching C compilers. There are two popular ways to ensure synchronicity between source and object:

  1. Make the object have the EXACT same timestamp as the source
  2. Cache some representation of the source with the object

In the former case, the algorithm for cache testing is this:

if  (timestamp(object file) == timestamp(source file))
    reuse object file
else
    recompile source file
    set timestamp(object file) = timestamp(source file)

In any POSIX environment, this timestamp setting can be accomplished with the stat() and utime() functions.

This exactitude of timestamps can be problematic when the source and object files are on different drives. For example, if one is on a network drive and the other on a local drive, and the two computers disagree on the current time, the timestamps may not match1.

A better solution is to keep a representation of the source along with the object. Copying the whole source is possible, but wasteful. Instead, a hash of the source can be computed and stored. Then, the next time you want to compile, recompute the hash (much faster than recompiling) and compare. If it matches, use the object; otherwise recompile. The fantastic ccache project uses MD4 hashes of C source code (after macro interpolation) to detect cache hits and misses.

Conclusions

This is an easy mistake to make. For the vast majority of users, Macromedia’s caching solution is a pure win. It’s only when you run into cases where timestamps do not increase monotonically that one encounters problems.

The best advice for programmers trying to implement a caching solution is be paranoid and consider your failure cases first. If you cannot judge your failure cases, or cannot afford any failure, you shouldn’t cache at all. In cases like compilation where a correct solution is paramount, correct-but-slow data is always better than fast-but-crappy data. This is even more true in financial settings. Imagine a scenario where ATMs cached data about account balances. All a gang of crooks would need to do is hit two ATMs at the same time. The same can go for e-voting machines without paper trails. In some cases, it’s better to just not cache because you can spend so much developer effort ensuring that the cache is good that you lose time making the process good in general.


1 This is a problem I’ve experienced with SMB. Generally, I try to allow a two-second timestamp window. If the machines are both NTP synched, then they should always agree to that level of precision, one hopes.