Firefox Extensions: Web Developer

This is the second installment of my exploration of useful Firefox extensions

One of the most popular and highly rated Firefox extensions is the Web Developer extension. This tool adds one or more of the following to every web page: a handy context menu or a toolbar. Both contain the same list of tools that let an experienced HTML coder investigate and manipulate the current page. The total list of features is too long to detail here (you really just have to try a few of them), but I’ll highlight some that I use personally.

My favorite feature is the one shown in the screenshot below. The “Validate Local HTML” uploads the page you are currently viewing to a validation service (by default, validator.w3.org). This permits you to test your HTML before uploading it to a public web server, or allows you to validate a page that is hidden behind a login or form submission.

A screenshot of the Web Developer context menu

Another handy feature for HTML debugging is the Outline control. This submenu allows you to specify any of a variety of HTML entities to highlight. The extension highlights them by drawing boxes around them.

The extension also has a few uses for general users. The Disable submenu has options to turn off aspects of annoying pages, like image animations, page colors or even entire styles for the worst-designed pages. The Forms submenu allows you to convert POST forms to GET, for cases when you want to tweak the form values in the URL bar, or bookmark the results of a form submission.

The Web Developer extension has quite an impressive array of tricks it can accomplish, all with standard operations. If nothing else it demonstrates how flexible and open a platform Firefox has become.

Next time: undoclosetab

Firefox Extensions: Introduction and Live HTTP Headers

Background

Interacting with the web has gotten more fun in the last year or so. Browser technological improvements seem to be getting a higher priority as Firefox, Safari and Opera are competing to present a superior, standards-compliant environment that benefits both developers and end-users. This means that developers can spend less time working around stupid bugs and more time creating interesting content.

These days, if you are building a web application a sound strategy is to create it initially under Firefox or Safari and then apply workarounds to IE problems at the end. It doesn’t really matter if you start with Firefox or Safari as anything that works under one is highly likely to work under the other (and any failure is more likely the developer’s fault, not the browser’s fault! yay!)

This is is marked contrast to five years ago when making a sophisticated website work in both Netscape and IE ususally meant deep sacrifices in functionality, or writing two separate websites.

Firefox in particular has caused an explosion of interest via its extension mechanism. Can you write Javascript and XML? If so, then you have the skills needed to customize the web experience. Because of this ease of development, the breadth of experimental new features has grown as the risk of trying out those features has shrunk. Of course, like the bulk of web itself many of the extensions are junk. But a few are winners that I can no longer imagine working without.

This post, and likely a few more that follow, will detail the Firefox extensions that I consider to be the most useful, primarily to web developers but also to general users. Eventually, I hope to also cover a few that are interesting, but not useful or good enough for regular use in my opnion.

And so, without further ado, I present…

Live HTTP Headers

Live HTTP Headers screen shot

This is the first Firefox plugin I ever installed and I still use it routinely today. Live HTTP Headers adds a new tab to the Page Info dialog that shows details of the HTTP request/response for that page. To see it, install the extension, visit a page and select Page Info (via the Tools menu, via the context menu, or via Cmd-I on Mac). Then select the Headers tab to see the HTTP headers panel.

This panel shows the full HTTP message exchanged between the browser and the web server, less the actual content. This data can be critical for debugging anything deeper than HTML/JS glitches. For example, I maintain the Perl module called CGI::Compress::Gzip which transparently compresses dynamic web content when the browser says that it can support it. I used the HTTP headers to determine what the browser was telling the web server regarding compression — in this case “Accept-Encoding: gzip” was the important one. Then I looked at the web server response, which would tell me the Content-Length (should be smaller when compressed!) and the “Content-Encoding: gzip” message.

The HTTP headers can also tell you what web server and platform is being employed at the other end. Finally, you get to see the raw cookie data that went over the wire.

On the command line, you can get easily see the response headers via something like

    wget -S -O /dev/null http://www.chrisdolan.net/

or

    curl -I http://www.chrisdolan.net/

but that’s not as real-world as viewing the browser transaction directly.

Viewing response headers can also be amusing. For example, visitors to Slashdot receive a random Futurama quote like the following:

    X-Bender: Farewell, big blue ball of idiots!

Next time: the Web Developer extension

When caches go bad

In general, caching is a good thing. Creating a temporary local copy of some data can dramatically speed up repeated access to that data, especially if it’s hard to recreate or slow to access. However, if the local copy falls out of sync with the real data it can sometimes be disastrous. Most of the hard work in caching is finding ways to ensure that the cached copy is fresh. This is called ensuring “cache coherency”.

For example, web browsers ask web servers if the remote content is newer than some time. If yes, then the cache is deleted and a new copy is fetched. If not, then the cache is used. This saves time because the simple “Is it fresh?” question is only a few bytes instead of the full size of the content. IE 5 had a painful misfeature that it didn’t send the “Is it fresh?” question for subsidiary files, like .css, .js, .jpg, etc files. So, even if the main .html page was up to date, sometimes IE used the wrong version of the supporting data.

Flash caches incorrectly

Last week I discovered a collosal caching problem with Flash MX 2004 (the authoring software, not the Flash Player that is in plugged into your browser). When switching to MX 2004, most users (me included) notice that compile time is dramatically improved. One of the many reasons is that Flash caches object files behind the scenes. That is, if you have a class defined in a .as file, Flash compiles it to a .aso file and stores that object file deep in a prefs directory. On subsequent compilations, if the .as file is older than the .aso file, the latter is reused, possibly saving several seconds.

Well, Macromedia screwed it up. By making the test be “Is it older?” they opened themselves up to a cache coherency problem. Consider this scenario:

Say, Alice and Bob are working on the same CVS project. Alice checks out some code one morning and starts programming. She makes some changes to Foo.as. Meanwhile, Bob speedily makes a critical bug fix to Bar.as and commits his change. An hour later, Alice finishes her Foo.as work and compiles her .fla. It fails because of the bug in Bar.as. Bob tells her, “I fixed that bug. Just do a cvs update and recompile.” She does, but the bug persists.

Why? It’s all about timestamps. Here’s what happened:

  • Alice checked out Bar.as with timestamp of 10:00 AM
  • Bob edited and checked in Bar.as with timestamp of 10:30 AM
  • Alice compiled at 11:00 AM, creating a Bar.aso file with a 11:00 AM timestamp
  • Alice cvs updated at 11:30 AM, getting Bar.as with a 10:30 AM timestamp
  • Alice recompiled her .fla, but Bar.aso is “newer” than Bar.as, so Flash ignored Bar.as and used the buggy Bar.aso instead.

So, Alice is stuck with an out-of-date .aso file that Flash won’t fix.

Workarounds

There are two easy workarounds to this problem, but both of them require manual intervention.

  1. Touch the .as file

    By editing the .as files, you make it newer again and Flash rebuilds the .aso. Instead of manually editing, the Unix “touch” command comes in handy. Here’s a one-line command to touch all of your project .as files to ensure they are newest:

    find . -name '*.as' | xargs touch
    

    If you do a lot of Actionscript work, though, this is a weak workaround becuase you may have updated .as files somewhere else in your classpath.

  2. Blow away the .aso files

    Like in the browser world, the answer here is “Clear your cache!” By deleting the .aso files you can force Flash to rebuild them all, at the expense of compile time. The trick is that you need to know where they are. Here’s a command that works on Mac OS X only (all one line):

    find ~/Library/Application*Support/Macromedia/Flash*MX*2004/en/Conf*/Classes \
        -name '*.aso' | perl -lpe 'unlink $_'
    

You have to remember to run one of these every time you have a case where you might introduce an older-but-changed .as file (often from a source repository like CVS or from a network drive).

What Macromedia should do

How can Macromedia fix this problem? Very simply! This problem was solved by caching C compilers. There are two popular ways to ensure synchronicity between source and object:

  1. Make the object have the EXACT same timestamp as the source
  2. Cache some representation of the source with the object

In the former case, the algorithm for cache testing is this:

if  (timestamp(object file) == timestamp(source file))
    reuse object file
else
    recompile source file
    set timestamp(object file) = timestamp(source file)

In any POSIX environment, this timestamp setting can be accomplished with the stat() and utime() functions.

This exactitude of timestamps can be problematic when the source and object files are on different drives. For example, if one is on a network drive and the other on a local drive, and the two computers disagree on the current time, the timestamps may not match1.

A better solution is to keep a representation of the source along with the object. Copying the whole source is possible, but wasteful. Instead, a hash of the source can be computed and stored. Then, the next time you want to compile, recompute the hash (much faster than recompiling) and compare. If it matches, use the object; otherwise recompile. The fantastic ccache project uses MD4 hashes of C source code (after macro interpolation) to detect cache hits and misses.

Conclusions

This is an easy mistake to make. For the vast majority of users, Macromedia’s caching solution is a pure win. It’s only when you run into cases where timestamps do not increase monotonically that one encounters problems.

The best advice for programmers trying to implement a caching solution is be paranoid and consider your failure cases first. If you cannot judge your failure cases, or cannot afford any failure, you shouldn’t cache at all. In cases like compilation where a correct solution is paramount, correct-but-slow data is always better than fast-but-crappy data. This is even more true in financial settings. Imagine a scenario where ATMs cached data about account balances. All a gang of crooks would need to do is hit two ATMs at the same time. The same can go for e-voting machines without paper trails. In some cases, it’s better to just not cache because you can spend so much developer effort ensuring that the cache is good that you lose time making the process good in general.


1 This is a problem I’ve experienced with SMB. Generally, I try to allow a two-second timestamp window. If the machines are both NTP synched, then they should always agree to that level of precision, one hopes.

Working with UTF-8 on OSX

One of my recent projects has been to port a Clotho application to Japanese. We chose this opportunity to convert the code base fully to Unicode. The other obvious choice would have been to port to Shift-JIS, a popular encoding for Japanese characters. However, we decided that this latter choice was not forward-looking enough. Choosing Unicode, specifically the UTF-8 encoding, will allow us to port to other languages more easily.

This article is an accumulation of my learnings configuring my Mac to default to UTF-8 wherever possible. Most Mac OS X apps default to ISO-8859-1 (aka Latin-1) in the English locale that I use. Fortunately, most of the common utilities have preferences to change that default, although the prefs are different for each App.

My Unix command line setup is still based on the default “C” locale (aka ASCII). In a future post I hope to explore switching that to UTF-8 as well.

General

Under OSX, Safari controls the general encoding prefs for the whole system.

  • open Safari Preferences
  • select Appearance tab
  • change Default Encoding to “Unicode (UTF-8)”

Firefox

  • open Firefox Preferences
  • select General tab
  • click Languages button
  • change Default Character Encoding to “Unicode (UTF-8)”

TextEdit

  • open TextEdit Preferences
  • change the Open pulldown from “Automatic” to “Unicode (UTF-8)”
  • change the Save pulldown from “Automatic” to “Unicode (UTF-8)”

When in doubt, open files with Cmd-O instead of double-clicking them

BBEdit

Warning! BBEdit 6.5 has a bug that files opened via drag or double-click ignoring the Open settings. When working with non-ASCII files, use Cmd-O. I have not tested any BBEdit versions newer than v6.5

  • open BBEdit Preferences
  • select Text Files: Opening
  • check “Translate Line Breaks”
  • set “Interpret File Contents as:” to “UTF-8”
  • check “Warn of Malformed UTF-8 Files”
  • select Text Files: Saving
  • set “Default Line Breaks:” to “Unix”

Terminal

  • open Terminal Window Settings
  • select the Display options
  • check “Wide glyphs for Japanese/Chinese/etc.”
  • check “Wide glyphs count as 2 columns”
  • change “Character Set Encoding:” to “Unicode (UTF-8)”
  • click Use Settings as Defaults

Flash

(this is for Flash MX 2004)

  • open Flash Preferences
  • select the ActionScript tab
  • change Open/Import to “UTF-8”
  • change Save/Export to “UTF-8”

Warning! Flash does NOT obey this preference for files created outside of Flash! Flash only interprets files as Unicode if they have a Byte-Order Mark (BOM). However, BOMs are a pain in plain text files since they break some editors.

As a workaround we recommend putting all non-ASCII dynamic text in external XML files instead of directly in the ActionScript.

Emacs

Upgrade to v21.3. This is newer than the version that is supplied by Apple for either 10.3 or 10.4. Fink provides this version (only in unstable as of this writing).

Add the following to your ~/.emacs file:

(setq locale-coding-system 'utf-8)
(set-terminal-coding-system 'utf-8)
(set-keyboard-coding-system 'utf-8)
(set-selection-coding-system 'utf-8)
(prefer-coding-system 'utf-8)