RFC: lethal - lexical exceptions for Perl
I've been scratching a long-time itch of mine, which is that while Perl allows me to use Fatal to get subroutines and built-ins to throw exceptions on failure, such a change was always done on a package-wide, rather than lexical (block) basis.
However with Perl 5.10 we have the ability to fix that, and I've put together a proof-of-concept module that enables exception-throwing subroutines on a lexical basis.
I'm looking for input into its naming, interface, and concept in general. Rather than filling up a blog post with it, you can read the node on PerlMonks.
trac-admin /path/to/tracdb
resync
Blocking Firefox 3: 206 bugs found.
Blocking Gecko 1.9: 175 bugs found.
Bug 421412 Get rid of _adjustWidth hack. Patch by Edward Lee.
Bug 396548. Star icon in url bar autocomplete is partly or completely buried under scrollbar (covered, hidden). Patch by Edward Lee.
Bug 406373. Url bar results can be scrolled horizontally after scrolling vertically. Patch by Edward Lee.
Bug 421490. wpad may be dangerous, and it is on by default. Patch by Michael Ventnor.
Bug 415329. large toolbar buttons are vertically misaligned. Patch by Dão Gottwald.
Bug 417421. Loss of back forward buttons when switching between 1.8 and 1.9. Patch by Dão Gottwald.
Bug 405010. make column changes in places/library (session) persistent. Patch by Marco Bonardo.
Bug 416225. Change “Unfiled bookmarks” to “Unsorted Bookmarks”. Patch by Kurt (supernova_00).
Bug 409215. Closing tabs quickly in succession causes the close button to “get stuck” (Minefield 3.0pre3, browser.tabs.closeButtons = 1). Patch by Gavin Sharp.
Bug 412360. Download Manager remains empty, downloads don’t start, with this 3.0b2 downloads.sqlite. Patch by Edward Lee.
Bug 418961. “Save Page As” “Text Files” saves file but Downloads window doesn’t show completion. Patch by Edward Lee.
Bug 421058. Firefox won’t prompt to change stored passwords. Patch by Justin Dolske.
No.
GCC Dehydra allows us to do analysis passes on our existing code. In the far future it may also allow us to do optimization passes. But it does not have the ability to do code rewriting, and probably won’t gain that ability any time soon. In order to be able to do C++->C++ transformations, you have to have correct end positions for all statements/tokens, not just beginning positions. GCC does not track this information, and making it do so would be a massive undertaking.
Mozilla2 still lives in a dual world where automatic code refactoring is done using the elsa/oink toolchain, while static analysis is taking place using the GCC toolchain.
Some days before, Reuben Thomas (a known contributor to Lua community) announced the release of Nancy, "a simple web site builder" at the Lua mailing list.
For the dismay of Lua afficcionados, this release was not written in Lua.
Oh, and it's written in Perl. I thought it might be interesting to say why. The problem was not libraries (I used my pure-Lua stdlib and LuaFileSystem), but deployment on Windows (I use Linux). I had previously built an all-in-one binary for Windows using a cross-compiler. That worked fine but made it hard to debug on Windows as I couldn't just edit-run, and also broke nastily when the cross-compiler was upgraded. So, I switched to LuaBinaries. That enabled me in the end to deploy only pre-built binaries, but at the cost of having to install 2 binary packages, and then do manual fiddling to get .lua files run as commands by the Windows shell.
ActivePerl does all this for me, and I was already using some Perl scripts in the same system, so by rewriting Nancy in Perl I got down to "run the Perl installer" as being the prerequisites for running Nancy.
Nancy begun its life as Lua code (and that's the reason Reuben announced this at Lua mailing list for the last time — unless it be written in Lua again in the future).
Perl still has an edge ahead before some of the popular dynamical languages out there, which (I hope) is not going away anytime soon in the future.
Filed under: TUAW Business, Podcasts
If you haven't checked out last week's show yet, by all means grab a copy. We were joined by Craig Hockenberry and Gedeon Maheux from Iconfactory, who gave us the lowdown on the history of the company, the origins of Twitterrific, the coevolution of Twitter with the now-dominant Mac client, and the promise of the upcoming iPhone development explosion. Download direct, listen in your browser or subscribe to the TalkShoe feed in iTunes.Continue reading Talkcast reminder, 10 pm ET tonight
Read | Permalink | Email this | CommentsFiled under: iTS
iTunes users who also happen to be fans of a goblin David Bowie, Gelflings, Skeksis, Mystics and/or Jim Henson have reason to celebrate: two of Henson's cult classic films are now available on iTunes. The Dark Crystal and Labyrinth are now available for purchase at the iTunes Store. This comes on the heels of the Farscape and Fraggle Rock iTunes releases in January.
If you've never been able to get the cry of the Skeksis or the image of a frighteningly glam Goblin King (with fantastic hair) out of your head, pop up iTunes and make some Jim Henson magic. While you're there, you can even grab the slightly watered-down musical numbers in the Labyrinth Soundtrack to go with it, as well as a copy of The Muppet Christmas Carol, just in case the kids need some lighter fare after The Dark Crystal. Can't beat that, right?
Read | Permalink | Email this | CommentsFiled under: Cult of Mac
I just now had to clean up some tables in a PostgreSQL database. The prior DBA thought that it would be good to split up tables into lx1, lx2, lx3 up to lx20. After I combined all the tables together, I needed to drop the originals. I could have written a Perl program to generate a series of drop table lx1; commands to feed into the psql command-line client, but instead I used the seq tool:
$ seq -f'drop table lx%g;' 1 20 drop table lx1; drop table lx2; ... drop table lx20;
If you don't have seq on your system, as on Mac OS X, you probably have jot, as in:
jot -w'drop table lx%g;' 20 1
Then again, if you just have to do it in Perl:
perl -le'print qq{drop table lx$_;} for 1..20'
but I like to use other tools than the Swiss Army Chainsaw sometimes.
Read more of this story at Slashdot.
Read more of this story at Slashdot.
Read more of this story at Slashdot.
So G-Archiver is a Windows app that lets you download your archived Gmail messages. It ends up that the app emails your Gmail username and password to the author. Crazy.
Just in case the power of the Firefox platform wasn't obvious already, check out the Web Slices work from Daniel Glazman and the Activities work Mike Kaply's doing.
"We're going to make sure that the JVM offers the Java applications as much access to the native ...
Only one iPhone application...
Read more of this story at use Perl.
Read more of this story at use Perl.
Read more of this story at Slashdot.
Read more of this story at Slashdot.
$.99 or $1.99 is indeed low, but I think Jens Alfke may have a point. It’s not that bigger, serious apps will sell for prices that low, but smaller, simpler apps that might otherwise have been released for free might generate real money with “cup of coffee”-level prices. It’s the App Store that makes this possible.
John Siracusa:
Skilled Mac developers are uniquely positioned to be the first to market with the iPhone applications they’ve been designing in their heads since last year. They know the tools, they know the technology, they even know a lot of the APIs already, and those they don’t know look a lot like the ones they do.
Do you remember Citizens' Band radio? Though established by the Federal Communications Commission in the 1950s, CB radio didn't become an overnight sensation until the 1970s when Moore's Law brought down the cost of radios to where it was economically viable to buy them solely for the purpose of breaking speed-limit laws. President Nixon, who liked to wear a blue suit and keep a cozy fire burning in his White House hearth year round no matter what the outside temperature or impact on his (our) air conditioning bill, had decided we all should drive 55 miles per hour or less to save fuel following the energy crisis of 1973. So, being true Americans, which is to say cranky and prone to complain, we en masse set out to break this new law using as our primary tool CB radio technology to warn us where Smokey was or had recently been or whether there was an eye in the sky. Criminals bound by a criminal code, we flaunted CB license restrictions (you were supposed to use your Federally assigned call sign from that license you were also supposed to have but never got) and operated under handles like "Thunderchicken" and "Boot-licker." I was "asciiboy." CB radio sales went from zero to tens of millions of units in under two years -- the highest rate of technology adoption ever seen in the U.S. before or since. Soon there was CB lore and a CB culture. CB was everywhere. When not breaking the law with it we were using CB as a huge social network to find the cheapest gas, the best hamburger or even a date for the prom. And then, quick as it started, CB was gone, worn to the bone from overuse and abuse and left to the truckers as it should have been all along. What killed CB radio was that moment when its annoyance factor exceeded its utility -- a utility already driven down by low traffic conviction rates and the eventual understanding that if everyone were a speeder then most cops wouldn't stop anyone.
I am beginning to think that Internet social networking is another CB radio, destined to crash and burn.
Social networking has a lot of problems as both a business and a cultural phenomenon. To start with there is generally no true business model. This can vary a bit from application to application but most are vying simply for eyeballs and hoping for Google ads to pay the bills until Time Warner or News Corp make them an acquisition offer they can't refuse. That might be okay for Facebook or MySpace and maybe Linked-In, but there are more than 350 general-purpose social networks out there and I will guarantee you that no more than 5 percent of those will be still operating two years from today.
If you are a social networking entrepreneur and your operation isn't among the top 10, I'd be either looking frantically for an acquirer or turning your site into a social networking aggregator, linking to many others in exactly the way the chat networks appear to be merging while still retaining their individual identities.
Then there is the annoyance factor, which for me is rapidly accelerating as the major social networks try to establish themselves as hosts for third-party applications. This would appear to be a no-brainer tactic for the two or three social networks that are likely to survive. In fact I could argue that what is more likely to survive than most social networks are the truly compelling applications that run upon them, eventually subsuming their hosts. But in the meantime there is all this annoying crap. How many groups do you have to join, how many causes do you have to support, how many silly applications do you have to run until you come to realize that you are being included TO DEATH?
My idea for the perfect Facebook app, for example, is one I call "I've Fallen and I Can't Get Up!" It's a variation on Twitter that is activated ONLY when one participant needs other participants to call 9-1-1 on his or her behalf. Maybe it could be linked to a panic button or to your cardiac pacemaker. The perfect Facebook application, then, is one you hope you'll never have to use. This is far better than the typical Facebook app, which is overused to the point where people withdraw or simply stop noticing.
It's not that I don't see value to social networks, it's that I generally don't see ENOUGH value. Yes, keeping my address book synchronized with reality is nice, but isn't that likely to be shortly absorbed into the operating system or perhaps into networked applications like Gmail and Yahoo Mail?
This trend has happened over and over as hundreds of portals came and went, leaving a few survivors. Same for hundreds of search engines, hundreds of free e-mail services, etc., etc.
Marshall McLuhan argued that obsolete communication technologies survive as art forms. This is true, I'd say, for Morse code and movable type printing and perhaps even for your venerable Rolodex or typewriter. But it isn't yet true for CB radio, nor for most Internet technologies. Maybe they aren't old enough yet to be appreciated. In the case of CB I think range of reception limits the possible population of players to something less than an artistic critical mass.
What will likely happen to social networking is that some applications will survive on a more modest basis than now (used by the trucker equivalents), others will morph into some new Next Big Thing as their more compelling sub-applications take over, and true hard-core social networkers will jump to more advanced technologies that eliminate the riff-raff. In the meantime, 70 percent or so of most social networking functionality -- the really useful functionality -- will be sucked into the dominant portal/search/e-mail/chat/social networks like MSN and Yahoo.
This next transition will happen faster than most people realize. Part of this is because Internet product cycles have been shortening for the past several years, so each generation is shorter than the one before. This hasn't mattered much because the audience has continued to expand. And even now as Internet growth in terms of new users is slowing, that's more than made up for by the shift of advertising budgets from print and broadcast to the net. So while the growth in users is decreasing, the growth in total revenue PER user is increasing. But so is the competition, hence the shorter product cycles.
The tip-off that we're nearing the end of a cycle is the flight to quality we're seeing from some of the bigger players. At Facebook, for example, you can no longer register using an e-mail address from an anonymous mail site like Mailinator, Operamail, or Countermail. Facebook demands that you take an extra three minutes and get a Yahoo Mail or AOL mail address for example. This is clearly the company pruning its subscribers in anticipation of an acquisition in the next couple quarters. There is no other reason to do it. MySpace isn't doing it despite a very real sex offender scandal, but then MySpace has already been sold and Facebook hasn't yet.
Once Facebook has been taken and one or two others, the golden era of social networking acquisitions will be over and the entrepreneurs will be headed for that Next Big Thing.
"Breaker Port 80! Do you have your ears on?"
This update installs software that enables remote disc sharing, system software restoration, and wireless migration with MacBook Air.
Read more of this story at Slashdot.
Trying to figure out the source of a method in one of our Catalyst controllers. Our controller inherited from one of our controller base classes. That, in turn, inherited from the deprecated Catalyst::Base which, in turn, does nothing but inherit from Catalyst::Controller, which inherits from three different classes. Two of those inherit from two classes each, which in turn ...
Let's see if I can figure this out our inheritance heirarchy so I can tell where that method is coming from:
Our::Controller
Our::ControllerBase
Catalyst::Base (deprecated passthrough)
Catalyst::Controller
Catalyst::Component
Class::Accessor::Fast
Class::Accessor
Class::Data::Inheritable
Catalyst::AttrContainer
Class::Accessor::Fast
Class::Accessor
Class::Data::Inheritable
Class::Accessor::Fast
Class::Accessor
Catalyst::Controller::REST
Catalyst::Controller
Catalyst::Component
Class::Accessor::Fast
Class::Accessor
Class::Data::Inheritable
Catalyst::AttrContainer
Class::Accessor::Fast
Class::Accessor
Class::Data::Inheritable
Class::Accessor::Fast
Class::Accessor
Got that?
On the off chance that you're a Multiple Inheritance fanboy, I don't think I'm going to say anything right now. I'm on the verge of profanity. MI is a tool of last resort (no, I'm not saying it's always the wrong answer). Today there are so many excellent alternatives that you really have no excuse for using MI other than "I don't like change".
Read more of this story at use Perl.
Net::HTTPServer does not handle large responses. To enable this support by breaking up large responses, change the _send function within: if (ref($data) eq "") { ...
to: if (ref($data) eq "") { if (length($data) > 1024) { my @data_chunks = unpack "a1024" x ((length($data)/1024)-1) . "a*", $data; foreach my $chunk (@data_chunks) { $self->_debug("SEND","_send: $chunk"); return unless defined($self->_send_data($sock,$chunk)); } } else { $self->_debug("SEND","_send: $data"); return unless defined($self->_send_data($sock,$data)); } }
See also: Regexp::Common::Email::Address.
If you didn't think think IE 8's "activities" were a derivative of Mozilla's microformats work then check out Mike Kapley's add-on.
Six months ago I had to set up a Linksys Access Point (WAP54G) as a wireless repeater in my house, to get the wireless signal from my Wireless-G Broadband Router (WRT54G) to reach the attic. It was a painful process, and the main reason it was painful was trying to get the correct value on the repeater for the setting labelled "Remote Access Point's LAN MAC Address". This is how it knows where to send all the relayed traffic to.
How hard can this be? You go to your WRT54G router, click "Status", and there it is: a field labelled "MAC Address". Just copy that in, right? However, that's the MAC address for the wired side. So, despite the fact that the field is labelled "LAN MAC address", if you enter the MAC address the router uses on the LAN, nothing works! It's not the "LAN MAC Address" at all - it's precisely the other one, the MAC address from the Status/Wireless subtab. Doh! A "Breathtaking UI" award to Linksys, I think.
(This is a bit geeky, but I also want to document it to try and avoid anyone else having to suffer through this.)
Adam Kaplan has released a cool new profiling tool, Devel::NYTProf. It's apparently taken the formatting beauty of Devel::Cover, the code test coverage tool, and used it to identify hotspots in your code. The results are beautiful: Overview and module-level detail. There's also a report writer that provides CSV output and lets you create your own output in your own format.
I ran into a divide-by-zero error that I patched in my local copy, but other than that the results seem good. I look forward to whatever other improvements come. I'm also glad it's been released with the support of the New York Times.
We have it on good authority (meaning this is a rumor, and “good authority” could mean anything) that...
As I previously threatened, I’ve gone ahead and created a “Microplanet” for Irish twitterers, similar to Portland’s Pulse of PDX — an aggregator of the “stream of consciousness” that comes out of our local Twitter community: IrishPulse.
Here’s what you can do:
Add yourself: if you’re an Irish Twitter user, follow the user ‘irishpulse’. This will add you to the sources list.
Publicise it: feel free to pass on the URL to other Irish Twitter users, and blog about it.
Read it: bookmark and take a look now and again!
In terms of implementation, it’s just a (slightly patched) copy of Venus and a perl script using Net::Twitter to generate an OPML file of the Twitter followers. Here’s the source. I’d love to see more “Pulse” sites using this…
More companies are showing their support for open source projects, and I couldn't be happier about it.
Those of you following Ovid's blog on use.perl.org, or reading his code improvements in the perl-qa mailing list, should give thanks to the BBC for supporting his Perl work. It's not all philanthropic, of course, since the BBC wants good tools for themselves, but I love that they're letting Ovid hitch his stories to the BBC wagon. That helps give Perl some credence in the eyes of open source skeptics.
Now, as you readers of Mechanix know, Devel::NYTProf is the hot new profiler in town. Not only is the New York Times allowing code to be released, it turns out there's a blog, open.blogs.nytimes.com, where Adam Kaplan announced the module. I love that a company that's not (exactly) in the software business is blogging about their open source software work. Let's hope it's a light in the darkness that others will lend their illumination to as well.
Since I moved house, I have all sorts of internet-related problems that I didn't have before. I used to do business with a small ISP, and I ran my own web server, my own mail service, and so on. When something was wrong, or I needed them to do something, I called or emailed and they did it. Everything was fine.
Since moving, my ISP is Verizon. I have great respect for Verizon as a provider of telephone services. They have been doing it for over a hundred years, and they are good at it. Maybe in a hundred years they will be good at providing computer network services too. Maybe it will take less than a hundred years. But I'm not as young as I once was, and whenever that glorious day comes, I don't suppose I'll be around to see it.
One of the unexpected problems that arose when I switched ISPs was that Verizon helpfully blocks incoming access to port 80. I had moved my blog to outside hosting anyway, because the blog was consuming too much bandwidth, so I moved the other plover.com web services to the same place. There are still some things that don't work, but I'm dealing with them as I have time.
Another problem was that a lot of sites now rejected my SMTP connections. My address was in a different netblock. A Verizon DSL netblock. Remote SMTP servers assume that anybody who is dumb enough to sign up with Verizon is also too dumb to run their own MTA. So any mail coming from a DSL connection in Verizonland must be spam, probably generated by some Trojan software on some infected Windows box.
The solution here (short of getting rid of Verizon) is to relay the mail through Verizon's SMTP relay service. mail.plover.com sends to outgoing.verizon.net, and let outgoing.verizon.net forward the mail to its final destination. Fine.
But but but.
If my machine sends more than X messages per Y time, outgoing.verizon.net will assume that mail.plover.com has been taken over by a Trojan spam generator, and cut off access. All outgoing mail will be rejected with a permanent failure.
So what happens if someone sends a message to one of the 500-subscriber email lists that I host here? mail.plover.com generates 500 outgoing messages, sends the first hundred or so through Verizon. Then Verizon cuts off my mail service. The mailing list detects 400 bounce messages, and unsubscribes 400 subscribers. If any mail comes in for another mailing list before Verizon lifts my ban, every outgoing message will bounce and every subscriber will be unsubscribed.
One solution is to get a better mail provider. Lorrie has an Earthlink account that comes with outbound mail relay service. But they do the same thing for the same reason. My Dreamhost subscription comes with an outbound mail relay service. But they do the same thing for the same reason. My Pobox.com account comes with an unlimited outbound mail relay service. But they require SASL authentication. If there's SASL patch for qmail, I haven't been able to find it. I could implement it myself, I suppose, but I don't wanna.
So far there are at least five solutions that are on the "eh, maybe, if I have to" list:
It also occurred to me in the shower this morning that the old ISP might be willing to sell me mail relaying and nothing else, for a small fee. That might be worth pursuing. It's gotta be easier than turning qmail-remote into a SASL mail client.
The serialmail thing is worth a couple of sentences, because there's an autoresponder on the qmail-users mailing-list that replies with "Use serialmail. This is discussed in the archives." whenever someone says the word "throttle". The serialmail suite, also written by Daniel J. Bernstein, takes a maildir-format directory and posts every message in it to some remote server, one message at a time. Say you want to run qmail on your laptop. Then you arrange to have qmail deliver all its mail into a maildir, and then when your laptop is connected to the network, you run serialmail, and it delivers the mail from the maildir to your mail relay host. serialmail is good for some throttling problems. You can run serialmail under control of a daemon that will cut off its network connection after it has written a certain amount of data, for example. But there seems to be no easy way to do what I want with serialmail, because it always wants to deliver all the messages from the maildir, and I want it to deliver one message.
There have been some people on the qmail-users mailing-list asking for something close to what I want, and sometimes the answer was "qmail was designed to deliver mail as quickly and efficiently as possible, so it won't do what you want." This is a variation of "Our software doesn't do what you want, so I'll tell you that you shouldn't want to do it." That's another rant for another day. Anyway, I shouldn't badmouth qmail-users mailing-list, because the archives did get me what I wanted. It's only a stopgap solution, and it might turn out to be a big mistake, but so far it seems okay, and so at last I am coming to the point of this article.
I hacked qmail to support outbound message rate throttling. Following a suggestion of Richard Lyons from the qmail-users mailing-list, it was much easier to do than I had initially thought.
Here's how it works. Whenever qmail wants to try to deliver a message to a remote address, it runs a program called qmail-remote. qmail-remote is responsible for looking up the MX records for the host, contacting the right server, conducting the SMTP conversation, and returning a status code back to the main component. Rather than hacking directly on qmail-remote, I've replaced it with a wrapper. The real qmail-remote is now in qmail-remote-real. The qmail-remote program is now written in Perl. It maintains a log file recording the times at which the last few messages were sent. When it runs, it reads the log file, and a policy file that says how quickly it is allowed to send messages. If it is okay to send another message, the Perl program appends the current time to the log file and invokes the real qmail-remote. Otherwise, it sleeps for a while and checks again.
The program is not strictly correct. It has some race conditions. Suppose the policy limits qmail to sending 8 messages per minute. Suppose 7 messages have been sent in the last minute. Then six instances of qmail-remote might all run at once, decide that it is OK to send a message, and send one. Then 13 messages have been sent in the last minute, which exceeds the policy limit. So far this has not been much of a problem. It's happened twice in the last few hours that the system sent 9 messages in a minute instead of 8. If it worries me too much, I can tell qmail to run only one qmail-remote at a time, instead of 10. On a normal qmail system, qmail speeds up outbound delivery by running multiple qmail-remote processes concurrently. On my crippled system, speeding up outbound delivery is just what I'm trying to avoid. Running at most one qmail-remote at a time will cure all race conditions. If I were doing the project over, I think I'd take out all the file locking and such, and just run one qmail-remote. But I didn't think of it in time, and for now I think I'll live with the race conditions and see what happens.
So let's see? What else is interesting about this program? I made at least one error, and almost made at least one more.
The almost-error was this: The original design for the program was something like:
One way to fix this is to have the processes append to the history file, but never remove anything from it. That is clearly not a sustainable strategy. Someone must remove expired entries from the history file.
Another fix is to have the read and the update in the same critical section:
Cleaning the history file could be done by a separate process that periodically locks the file and rewrites it. But instead, I have the qmail-remote processes to it on the fly:
Here's a mistake that I did make. This is the block of code that sleeps until it's time to send the message:
while (@last >= $msgs) { my $oldest = $last[0]; my $age = time() - $oldest; my $zzz = $time - $age + int(rand(3)); $zzz = 1 if $zzz 1; # Log("Sleeping for $zzz secs"); sleep $zzz; shift @last while $last[0] < time() - $time; load_policy(); }The throttling policy is expressed by two numbers, $msgs and $time, and the program tries to send no more than $msgs messages per $time seconds. The @last array contains a list of Unix epoch timestamps of the times at which the messages of the last $time seconds were sent. So the loop condition checks to see if fewer than $msgs messages were sent in the last $time seconds. If not, the program continues immediately, possibly posting its message. (It rereads the history file first, in case some other messages have been posted while it was asleep.)
Otherwise the program will sleep for a while. The first three lines in the loop calculate how long to sleep for. It sleeps until the time the oldest message in the history will fall off the queue, possibly plus a second or two. Then the crucial line:
shift @last while $last[0] < time() - $time;which discards the expired items from the history. Finally, the call to load_policy() checks to see if the policy has changed, and the loop repeats if necessary.
The bug is in this crucial line. if @last becomes empty, this line turns into an infinite busy-loop. It should have been:
shift @last while @last && $last[0] < time() - $time;Whoops. I noticed this this morning when my system's load was around 12, and eight or nine qmail-remote processes were collectively eating 100% of the CPU. I would have noticed sooner, but outbound deliveries hadn't come to a complete halt yet.
Incidentally, there's another potential problem here arising from the concurrency. A process will complete the sleep loop in at most $time+3 seconds. But then it will go back and reread the history file, and it may have to repeat the loop. This could go on indefinitely if the system is busy. I can't think of a good way to fix this without getting rid of the concurrent qmail-remote processes.
Here's the code. I hereby place it in the public domain. It was written between 1 AM and 3 AM last night, so don't expect too much.
The Acid 3 Test has been officially released. The test has been in development for some time, with much of that development happening in the open.
The Acid 3 test is far more complex than the Acid 2 test. It covers a wider range of standards and consists of many more individual tests. Browsers have to render a sequence of boxes that display dynamically in a stairstep pattern. For every cluster of tests passed successfully, the boxes will fill in with a color, which signifies that all of the tests covered by that block have passed.
If you run Acid 3 on the shipping versions of current browsers (Firefox 2, Safari 3, Opera 9, IE7), you’ll see that they all score quite low. For example Safari 3 scores a 39/100. This percentage score is a bit misleading however. The situation with all four browser engines really isn’t that bad.
You can think of the Acid 3 test as consisting of 100 individual test suites. In order for a browser engine to claim one of these precious 100 points, it has to pass a whole battery of tests around a specific standard. In other words it’s like the browser is being asked to take 100 separate exams and score an A+ on each test in order to get any credit at all.
The reality is that all of the browsers are doing much better than their scores would have you believe, since the engines are often passing a majority of the subtests and experiencing minor failures that cost them the point for that section.
Shipping Safari scores a 39/100 with some significant rendering errors. We’ve been working hard since the test surfaced and are pleased to report that we’ve entered the “A” range on the test with a score of 90/100.
So what did we fix to gain so many points?
Bug 17064 has all the details, but here are the highlights.
Support for CSS3 Selectors
We added support for all of the remaining CSS3 selectors. These include selectors like nth-child, nth-of-type, last-child, last-of-type, etc. These selectors were already implemented in KHTML, and the KHTML developers had even kindly provided patches for us in the relevant WebKit bugs. Therefore it was a simple matter of taking those patches, updating them to the WebKit codebase, and then merging them in. A big thanks to the KHTML developers for their hard work in this area.
Parsing Bugs
WebKit had a number of minor parsing bugs that Acid 3 targeted. The boxes did not render properly because of an obscure parsing bug that the test exploited (thanks, Hixie). In addition a number of other parsing bugs kept us from completely passing individual tests. We have updated our parser to be much closer to the HTML5-specified parsing rules.
WebKit has also never parsed DOCTYPEs before. I re-wrote WebKit’s DOCTYPE parsing to match the HTML5 specification, and so now if you put a DOCTYPE into your page it will be present in the DOM. In addition many bugs centered around proper mode resolution (quirks vs. strict) have now been fixed. You can document.write a DOCTYPE for example in a new document and have the correct mode be selected.
SVG
Acid3 has many SVG tests. We’ve been hard at work making these tests pass. In particular SVG font support and other aspects of the SVG DOM have been tested. Many of the remaining 10 points are SVG failures. We’ll be working on SVG animation in order to pass the last few SVG tests.
DOM
Acid3 tests a lot of DOM level 2 features, like traversal and ranges. It particularly focuses on the “liveness” of objects, e.g., making sure everything updates properly when you dynamically change a document by adding/removing nodes. Most of our failures in this area had to do with not behaving properly in the presence of these dynamic changes (even though we tended to pass the more static tests).
Now that we’re closing in on 100%, we’ll be blogging about each fix as it happens, so that you can follow our progress from the blog.
I do think the IE team deserves credit for having floating the idea for opt-in version targeting rather than just going ahead and implementing it.
Err, “floated the idea?” I thought what I read was an announcement of fait accompli. At no time did it strike me as though Microsoft left the issue open-ended. That they subsequently revoked their proclaimed decision came out of left field; would this have been the case if what they first did was in fact merely “floating the idea?”
I’m glad that someone inside Microsoft apparently somehow managed to overrule someone else (whoever the people involved are), but I can find no way to interpret Microsoft’s initial course of action as commendable.
Furthermore, the question Eric Meyer asked is still open: even though Microsoft reneged on the most objectionable part of the announcement, the fact that IE 8 will have three rendering modes, including a frozen-in-time one with all its implications for competitors, still stands.
A few days ago, I was writing a code (namely the Path-Classy dist) and stared at the code that produced the file size in raw bytes or "humanized" (28300 or 28K).
sub _format_size {
my ($sz, $opt) = @_;
my $format = $opt->{format};
if ( $format eq 'h' ) {
require Numbers::Bytes::Human;
return Number::Bytes::Human->new->format( $sz );
}
else { # raw bytes
return $sz;
}
}
Of course, loading Number::Bytes::Human
and creating a instance every time $f->size({ format => 'h' })
was invoked seemed overkill. But saving the N::B::H
into a class/instance variable seemed overkill too: it has nothing to do with Path::Classy
(which are Path::Class
) objects but for that instant relationship to format a file property, size.
Hey, that's a chance to use memoization, splitting the formatter creation into a function and then memoizing it (so that we don't need to create a [reusable] object with the same capabilities over and over), we come to this code.
use Memoize;
sub _size_formatter {
require Number::Bytes::Human;
return Number::Bytes::Human->new;
}
memoize('_size_formatter');
sub _format_size {
my ($sz, $opt) = @_;
my $format = $opt->{format};
if ( $format eq 'h' ) {
return _size_formatter->format( $sz );
...
That looked elegant to me. To make it even more tight (and to require yet another CPAN module ;-) ), using Attribute::Memoize
seemed right. It avoids the need to repeat the function name in the memoize
call and it anticipated the wrapping up of the sub to BEGIN time (a free bonus of Attribute::Handlers
in the backstage).
use Attribute::Memoize;
sub _size_formatter :Memoize {
require Number::Bytes::Human;
return Number::Bytes::Human->new;
}
That's it! Efficient code, localized behavior, no need for extra variables. Will people understand that for maintenance? I hope so.
Win32::File::Object->new('file.txt', 'autowrite')->readonly(0);
Dropping vowels to shorten names is a terrible practice. Quick, someone give me an idea what $hdnchgdsp means, an Actual Variable from some Bad Code I'm working on today.
It's not just variables names, either. Filenames often need to be shortened, but dropping vowels is not the way to do it. You're left with unpronounceable names that are annoying to type.
The key to effective abbreviation is not removal of letters from the middle of the words, but from the end. Sometimes, it doesn't make sense to shorten a word at all, like "post". If you have a file that is supposed to "post audit transactions", call it "post-aud-trans" or "post-aud-trx", not "pst_adt_trns".
Jott is a really neat service that lets you Do Stuff via your cell phone. The default Stuff you can do is "send email and SMS" and "setup a reminder." There's also a very simple API for writing your own applications (called Jott Links). It works something like this:
There are Jott Links for Twitter and other things that I don't care about. There isn't one for Hiveminder. Zak Greant made a video demonstrating Hiveminder and Jott together, which has Jott send mail to the task-by-email interface of Hiveminder. This isn't bad at all, but it puts all kinds of crap into your task, because Jott sends pretty chatty email.
I wrote a Jott Link service in about ten minutes (much of which was test time, waiting for Jott to transcribe my messages). It uses CGI.pm and Net::Hiveminder to create a very concise task. I need to add more features to it, but I'm in no rush. I am secretly hoping that the guys at Best Practical will write a much better version, complete with a user setup link, so that everyone can use their link, rather than running his own, each on a different server.
Here's my code:
#!/usr/bin/perl
use strict;
use warnings;
BEGIN { $ENV{HOME} = '/home/rjbs' }
use CGI qw(:standard);
use Net::Hiveminder;
my ($pw, $key) = `cat /home/rjbs/.hiveminder`;
chomp($pw, $key);
my $hm = Net::Hiveminder->new(
email => 'user@example.com',
password => $pw,
);
my $user_key = url_param('userKey');
my $message = url_param('message');
die unless lc $user_key eq lc $key;
$hm->create_task("$message\n via Jott.com");
print "Content-type: text/plain\n\nCreated.";
Obviously, this is a horrible hack. Still, it means I can open my phone, hold down 5, and dictate todo items right into Hiveminder.
There are a lot of little problems with Jott, some of which strike me as significant usability issues, but they're all very fixable, and I look forward to seeing them fixed. I'll write more about them later. Here's the one that irked me the most last night: Jott says that to write a Jott Link, you should expect an HTTP POST. You do, in fact, get a POST, but all of the data is in the URL query string, not in the content of the request. Huh?
Well, whatever. All their problems are fixable, and the service looks like it will be great.
I've just uploaded the new Test::Aggregate. The only change is that it sets the $ENV{TEST_AGGREGATE} variable to true (for VMS users: and explicitly deletes it in an END block). This allows you to do this:
die "Cannot use $0 in aggregated tests"
if $ENV{TEST_AGGREGATE};
Blew a chunk of time yesterday trying to debug why the aggregated tests were failing. Now I can add that to some code to ensure that I'll know immediately next time.
A couple of weeks ago, WebSense posted this article with details of a spammer’s attack on Google’s CAPTCHA puzzle, using web services running on two centralized servers:
[…] It is observed that two separate hosts active on same domain are contacted during the entire process. These two hosts work collaboratively during the CAPTCHA break process. […]
Why [use 2 hosts]? Because of variations included in the Google CAPTCHA image, chances are that host 1 may fail breaking the code. Hence, the spammers have a backup or second CAPTCHA-learning host 2 that tries to learn and break the CAPTCHA code. However, it is possible that spammers also use these two hosts to check the efficiency and accuracy of both hosts involved in breaking one CAPTCHA code at a time, with the ultimate goal of having a successful CAPTCHA breaking process.
To be specific, host 1 has a similar concept that was used to attack Live mail CAPTCHA. This involved extracting an image from a victim’s machine in the form of a bitmap file, bearing BM.. file headers and breaking the code. Host 2 uses an entirely different concept wherein the CAPTCHA image is broken into segments and then sent as a portable image / graphic file bearing PV..X file headers as requests. […]
While it doesn’t say as such, some have read the post to mean that Google’s CAPTCHA has been solved algorithmically. I’m pretty sure this isn’t the case. Here’s why.
Firstly, the FAQ text that appears on “host 1″ (thanks Alex for the improved translation!):
FAQ
If you cannot recognize the image or if it doesn’t load (a black or empty image gets displayed), just press Enter.
Whatever happens, do not enter random characters!!!
If there is a delay in loading images, exit from your account, refresh the page, and log in again.
The system was tested in the following browsers: Internet Explorer Mozilla Firefox
Before each payment, recognized images are checked by the admin. We pay only for correctly recognized images!!!
Payment is made once per 24 hours. The minimum payment amount is $3. To request payment, send your request to the admin by ICQ. If the admin is free, your request will be processed within 10-15 minutes, and if he is busy, it will be processed as soon as possible.
If you have any problems (questions), ICQ the admin.
That reads to me a lot like instructions to human “CAPTCHA farmers”, working as a distributed team via a web interface.
Secondly, take a look at the timestamps in this packet trace:
The interesting point is that there’s a 40-second gap between the invocation on “Captcha breaking host 1″ and the invocation on “Captcha breaking host 2″. There is then a short gap of 5 seconds before the invocations occur on the Gmail websites.
Here’s my theory: “host 1″ is a web service gateway, proxying for a farm of human CAPTCHA solvers. “host 2″, however, is an algorithm-driven server, with no humans involved. A human may take 40 seconds to solve a CAPTCHA, but pure code should be a lot speedier.
Interesting to note that they’re running both systems in parallel, on the same data. By doing this, the attackers can
collect training data for a machine-learning algorithm (this is implied by the ‘do not enter random characters!’ warning from the FAQ — they don’t want useless training data)
collect test cases for test-driven development of improvements to the algorithm
measure success/failure rates of their algorithms, “live”, as the attack progresses
Worth noting this, too:
Observation*: On average, only 1 in every 5 CAPTCHA breaking requests are successfully including both algorithms used by the bot, approximating a success rate of 20%. The second algorithm (segmentation) has very poor performance that sometimes totally fails and returns garbage or incorrect answers.
So their algorithm is unreliable, and hasn’t yet caught up with the human farmers. Good news for Google — and for the CAPTCHA farmers of Romania ;)
The Perl Foundation needs new blood. Jim Brandt writes:
Have you ever wanted to get involved in The Perl Foundation, but didn't know how? Well, now's your chance. I'm pleased to announce open self-nominations for the following TPF roles:
You can follow the links above to read descriptions of each of the positions. If you think you're a good fit for one or more of them, send me an email at cbrandt at perlfoundation dot org. I'll then invite you to a dedicated wiki we have set up just for the election.
Once you join the wiki, you'll set up a page to post all of your experience and answer the questions provided in each section above. The wiki is private, but you'll be able to see the other candidate pages, and they'll see yours.
The deadline to get all of your information in is midnight next Tuesday, March 11. Our committees elect their members, so the Conferences Committee will be voting on the CC chair and the Steering Committee will vote on the chair and PR positions. After we have a chance to look over everyone's information, we vote and select our newest members.
You only have a week, so don't wait too long. I look forward to hearing from you.
Karen Pauley is stepping up to run for Steering Committee chair, so how about you? Maybe that's a spot you'd like to work on, or maybe public relations is more up your alley. This is your chance to help lead TPF lead Perl and Perl development.
Astute followers of TPF will note that the PR spot is open, a spot that I once held. Yes, I am no longer doing PR for TPF. I've done that job for a while, and now I'm moving on to do other things, not least of which is this little news called perlbuzz.com.
Net::HTTPServer also enables client side certificate checking by default without any built in interface to change this.
I've added an option to new() to set this on the fly, but as a workaround see line 825 of HTTPServer.pm and change SSL_verify_mode to desired setting:
0x00 - no verification 0x01 - verify peer certificate 0x02 - fail verification if no peer certificate exists; ignored for clients 0x04 - verify client once
for more details on these options see: http://search.cpan.org/~sullr/IO-Socket-SSL-1.13/SSL.pm
Read more of this story at Slashdot.
Today, Best Practical announced IMAP access to Hiveminder. It's way cool, and I'm sure I'll end up making a lot of improvement to my mutt configuration tools to make the most of it. You can check out their blog post or documentation for more information, but basically you point your IMAP client at Hiveminder and you can see your todo list. You can drop new tasks (in the form of email from elsewhere) into inbound folders and you can move existing tasks into other folders to cause them to become hidden or complete. There's a bit more to it, but that's the gist.
My IMAP client of choice is OfflineIMAP, as I've said many times before. It's the easiest way for me to use mutt with IMAP, whether online or off. Unfortunately, it has a really stupid bug. Every message in an IMAP account has a unique id (the UID), which is useful for doing synchronization. It lets you figure out that you've moved a message from one place to another in your offline store. OfflineIMAP doesn't seem to keep the same UID on messages that have moved from one folder to another, which made it impossible to use the IMAP interface to mark a message done or hidden.
As usual, the guys at Hiveminder were quick to sort this out, making their correct software cope with my twitchy software.
Now, the folders in which Hiveminder presents your tasks are (I am told) great for users of GUI MUAs, where they form a nice hierarchy of folders that you can drill down through. Here's a summary of the folder layout:
Actions
Completed
Hide for
Days..
01 day
(..more..)
Months..
01 month
(..more..)
Take
Braindump mailboxes
[]
Groups
pep
All tasks
Everyone else's tasks
Up for grabs
Help
News
Here's what they look like as directories:
Actions/Completed
Actions/Hide for/Days../01 day
Actions/Hide for/Months../01 month
Actions/Take
Braindump mailboxes/[]
Groups/pep
Groups/pep/All tasks
Groups/pep/Everyone else's tasks
Groups/pep/Up for grabs
Help
News
The amount of typing needed to move things around between these folders is a drag. Fortunately, OfflineIMAP makes it really simple to map Hiveminder's IMAP folders into a nice, shallow, easy to type hierarchy. With my OfflineIMAP configuration, it looks like this:
./braindump
./braindump.[]
./done
./groups
./groups.pep
./groups.pep.all
. /groups.pep.avail
./groups.pep.others
./help
./hide.1d
./hide.1m
./inbox
. /take
I need to do a bit of work to make
WhichConfig check $0
(or
something) to notice that I want to use Hiveminder, rather than the "normal"
mail available to it. Even without having done that, the IMAP interface is
pretty fantastic. I see a lot of weird Maildir tricks in my future. Until I
have some to publish, here's my OfflineIMAP configuration for use with
Hiveminder:
.offlineimap
:
[general]
pythonfile = ~/.offlineimap/helper.py
[Account hiveminder]
localrepository = hiveminder_maildir
remoterepository = hiveminder_imap
[Repository hiveminder_imap]
type = IMAP
remotehost = hiveminder.com
ssl = yes
remoteuser = user@example.com
remotepass = PASSWORD
nametrans = lambda foldername: hm_nametrans(foldername)
folderfilter = lambda foldername: hm_folderfilter(foldername)
[Repository hiveminder_maildir]
type = Maildir
localfolders = ~/Mailhive
This relies on a few Python functions stored in another file:
helper.py
:
import re
hide_re = re.compile('^Actions/Hide')
spec_re = re.compile('(?P<n>\d\d) (?P<units>days?|months?)$')
def hm_folderfilter(folder):
if folder in ('Actions', 'Groups'): return False
if hide_re.search(folder) and not spec_re.search(folder): return False
return True
def hm_nametrans(folder):
if folder == 'Actions/Completed': return 'done'
if folder == 'Actions/Take': return 'take'
if folder == 'Actions/Take': return 'take'
folder = re.compile('Braindump mailboxes').sub('braindump', folder)
if hide_re.search(folder):
spec = spec_re.search(folder)
n = int(spec.group('n'))
units = spec.group('units')
return 'hide/%s%s' % (n, units[0:1])
if re.compile('^Groups').search(folder):
folder = re.compile('All tasks').sub('all', folder)
folder = re.compile('Up for grabs').sub('avail', folder)
folder = re.compile("Everyone else's tasks").sub('others', folder)
return folder.lower()
I've put up a quick site with the results of Acme::ReturnValue:
http://returnvalues.useperl.at
This page shows that Acme::ReturnValue is far from perfect because it contains a whole lot of false positives. But there are also some very funny things to be found.
I'm not sure if I'll spend much more time on Acme::ReturnValue, so I doubt that the website will see any improvements (filters, sorting, links to source code, ..). But I do plan to look at Perl::Critic::Policy::Modules::RequireEndWithOne, as suggested by Chris Dolan.
And I will set up a cronjob to generate the site (but probably only once a week - I want to spare those cycles for cpants...)
I've been a rather slack CPAN author for the last couple of years and haven't been updating my modules as much as I should.
But over the weekend, whilst updating the Teach-In slides for the UKUUG conference I discovered the new CPAN testers matrix and, in particular, the appalling test results for the current version of Symbol::Approx::Sub.
Obviously having major test failures in such a crucial module is a terrible state of affairs so I spent some time looking at the problem last night.
Luckily it was a relatively easy fix. It was just a test which had been written by someone (me!) whose knowledge of Perl had temporarily left them. Fixing it was simple enough, and I was able to release the first new version of Symbol::Approx::Sub for over two years. I also switched the distribution from ExtUtils::MakeMaker to Module::Build and made a few other tweaks which have improved its kwalitee.
And this morning, I saw a much happier set of test results. Which was nice.
Whilst poking around in my test results, I noticed that WWW::MakeAShorterLink was also failing far too many tests. It turns out that this is because the MASL web site no longer exists. Or, rather, it redirects to TinyURL. So I've submitted a deletion request for WWW::MakeAShorterLink. It will be removed from CPAN in the next couple of days. All of your URL-shortening needs will be handled by WWW::Shorten. And, yes, next on my list is to fix the problems in that distribution's test plan.
Many thanks must go to the CPAN testers whose is so important in revealing my inadequacies as a programmer.
As part of our ongoing effort to maintain our test suite's performance, I've just added the following quick hack to source control (it needs a lot of work):
#!/usr/bin/env perl
use strict;
use warnings;
use App::Prove::State;
use List::Util 'sum';
use Lingua::EN::Numbers 'num2en';
my $prove = '.prove';
unless (-f $prove && -r _) {
die "Cannot find or read $prove file";
}
my $state = App::Prove::State->new({ store => $prove });
my $generation = $state->{_}{generation};
my $tests = $state->{_}{tests};
my $total = sum(map { $_->{elapsed} } values %$tests);
my $minutes = int($total / 60);
my $seconds = int($total % 60);
my $num_tests = shift || 5;
if ($num_tests > keys %$tests) {
$num_tests = keys %$tests;
}
my $num_word = num2en($num_tests);
my %time_for;
while (my ($test, $data) = each %$tests) {
$time_for{$test} = $data->{elapsed};
}
my @sorted_by_time_desc
= sort { $time_for{$b} <=> $time_for{$a} } keys %time_for;
print "Generation $generation\n";
print "Total runtime approximately $minutes minutes $seconds seconds\n";
print "\u$num_word slowest tests:\n";
for (0 .. $num_tests) {
my $test = $sorted_by_time_desc[$_];
print "\t$time_for{$test} seconds -> $test\n";
}
Basically, if you use the '--state=save' option with prove, it will save the state of your tests in a .prove file. This code reads the file, tells you which generation the file is, total test suite run time in minutes and seconds and your 5 slowest tests. Pass it a number and it will tell you the X slowest tests.
Update. Here's the output from the current branch I'm working on (a slightly updated version from above):
Generation 18
Number of test programs: 58
Total runtime approximately 17 minutes 35 seconds
Five slowest tests:
482.732247114182 seconds -> t/acceptance.t
234.499103069305 seconds -> t/aggregate.t
96.313854932785 seconds -> t/standards/strict.t
66.6500070095062 seconds -> t/unit/db/migrations.t
56.7010760307312 seconds -> t/unit/piptest/pprove/testdb.t
13.5212490558624 seconds -> t/unit/api/builder/brand-promotions.t
... a logical negation function ... takes a boolean argument and returns a boolean result.I worried for some time about whether to capitalize "boolean" here. But writing "Boolean" felt strange enough that I didn't actually try it to see how it looked on the page.
I looked at the the Big Dictionary, and all the citations were capitalized. But the most recent one was from 1964, so that was not much help.
Then I tried Google search for "boolean capitalized". The first hit was a helpful article by Eric Lippert. M. Lippert starts by pointing out that "Boolean" means "pertaining to George Boole", and so should be capitalized. That much I knew already.
But then he pointed out a countervailing consideration:
English writers do not usually capitalize the eponyms "shrapnel" (Henry Shrapnel, 1761-1842), "diesel" (Rudolf Diesel, 1858-1913), "saxophone" (Adolphe Sax, 1814-1894), "baud" (Emile Baudot, 1845-1903), "ampere" (Andre Ampere, 1775-1836), "chauvinist" (Nicolas Chauvin, 1790-?), "nicotine" (Jean Nicot, 1530-1600) or "teddy bear" (Theodore Roosevelt, 1858-1916).Isn't that a great paragraph? I just had to quote the whole thing.
Lippert concluded that the tendency is to capitalize an eponym when it is an adjective, but not when it is a noun. (Except when it isn't that way; consider "diesel engine". English is what it is.)
I went back to my example to see if that was why I resisted capitalizing "Boolean":
... takes a boolean argument and returns a boolean result.Hmm, no, that wasn't it. I was using "boolean" as an adjective in both places. Wasn't I?
Something seemed wrong. I tried changing the example:
... takes an integer argument and returns an integer result.Aha! Notice "integer", not "integral". "Integral" would have been acceptable also, but that isn't analogous to the expression I intended. I wasn't using "boolean" as an adjective to modify "argument" and "result". I was using it as a noun to denote a certain kind of data, as part of a noun phrase. So it is a noun, and that's why I didn't want to capitalize it.
I would have been happy to have written "takes a boolean and returns a boolean", and I think that's the controlling criterion.
Sorry, George.
Have you ever wanted to get involved in The Perl Foundation, but didn't know how? Well, now's your chance. I'm pleased to announce open self-nominations for the following TPF roles:
You can follow the links above to read descriptions of each of the positions. If you think you're a good fit for one or more of them, send me an email at cbrandt at perlfoundation dot org. I'll then invite you to a dedicated wiki we have set up just for the election.
Once you join the wiki, you'll set up a page to post all of your experience and answer the questions provided in each section above. The wiki is private, but you'll be able to see the other candidate pages, and they'll see yours.
The deadline to get all of your information in is midnight next Tuesday, March 11. Our committees elect their members, so the Conferences Committee will be voting on the CC chair and the Steering Committee will vote on the chair and PR positions. After we have a chance to look over everyone's information, we vote and select our newest members.
You only have a week, so don't wait too long. I look forward to hearing from you.
Read more of this story at Slashdot.
I have a fairly complicated mutt configuration. It could probably do with more streamlining, but it's pretty easy for me to update, because of the way I generate it.
Some of it, of course, comes from Addex. That's only some of it, though. Namely, the folder hooks, folder subscriptions, and a lot of the aliases. That leaves a lot of things unconfigured: default headers, folder names and locations, connection methods, helper programs (i.e., the mailcap), OpenPGP and S/MIME config, keybindings, colors, and all sorts of other little things.
A lot of these I really do want to set myself, once, by hand. I want the same
basic colors everywhere, for example. A lot of other things vary from host to
host. On the server to which mail mail is delivered, my mail is laid out in
Courier-style Maildirs, so my inbox is in ~/Maildir/{cur,new,tmp}
and one of
my folders is ~/Maildir/cpan/{cur,new,tmp}
and so on. On my laptop, where I
sync that mail with OfflineIMAP, I end up with my inbox in
~/Maildir/INBOX/{cur,new,tmp}
. Some folders are renamed, others are omitted.
That affects save hooks, folder hooks, folder subscriptions, and other
settings.
At work, I have an entirely different set of folders, as well as a different From header, a different sig, and other different settings. On all my machines, I have different helper applications. On my workstation at work, which I use only via ssh, I can't open images. On my laptop, I need to use a wrapper around Preview.app. Elsewhere, I might want to use Firefox. On some machines, I want HTML rendered with lynx, and if there's no lynx, html2text will do, and so on.
So, a lot of my mutt configuration is generated for me by little programs, either when I run mutt or when I deploy its configuration to my homedir from a git checkout.
Since it's important to know what machine I'm running on, I have a module
called WhichConfig. It exports a routine, which_config
that tells me what
configuration to use. For the most part, this is either a hostname or
"default." It works like this:
ifconfig
, use the associated configThe ifconfig
check is pretty crude, but very, very effective. I look for the
MAC address on hosts for which I control the hardware and IP address on hosts
that I don't. It's easy to change the way the test works, but so far this has
always worked. Many of the rest of my tools rely on WhichConfig.
The first application is simple; my muttrc contains this line:
source `~/.mutt/which-config`
which-config
just prints out a filename based on the appropriate config.
That file contains settings that I just want to set by hand on each host.
A more interesting (but still very simple) helper script is folder-finder
.
It finds all the relevant maildirs for the current configuration and subscribes
to them. It knows how to find both Courier-style and OfflineIMAP-style folders
and normalize their names. It runs every time I run mutt, so that it will find
folders that have been recently created. Unfortunately, mutt can't include the
multi-line output of a script via the "source" directive or backticks
mechanism, so I end up with this in my muttrc:
source `~/.mutt/folder-finder > ~/.mutt-folders; echo ~/.mutt-folders`
This minor inelegance is definitely worth the benefits.
Finally, there's make-mailcap
. This program looks at WhichConfig and looks
for known useful commands in $PATH
, then reads in /etc/mailcap
and dumps an
output file into ~/.mutt/mailcap
. This file is used as the mailcap file,
which mutt uses to figure out how to view or print attachments. This is useful
for dumping HTML mail to text (when it has no useful text alternative), or for
opening pictures and PDFs in Preview, xv, or xpdf. Over time, I imagine I'll
add more and more helpers to make-mailcap
, but the most useful one is
osx-preview-img
. When mutt uses a helper program to open an attachment, it
generally sees a specification like "lynx -dump -force_html '%s'
" and
replaces the %s
with the name of a temporary file, which is deleted after the
command exits.
In MacOS X, the way to open a file in a GUI application is usually to use the
open
command, like this:
$ open -a Preview some-image.jpg
The problem is that open
exits nearly instantly, having send a request to
Preview. mutt then deletes the temporary file, and Preview sends a SIGWTF. My
helper program copies the tempfile to another location and has Preview open
that. It never cleans up that tempfile, but since it's under /tmp, I know the
OS will clean it up on reboot. In the future, I may look at using Mac::Glue to
write a helper that won't leave clutter in /tmp, but I'm not too worried about
it for now.
All of this stuff has been so useful to me that I feel like I should bundle it up and drop it on the CPAN, but I'm not sure whether (or how) anyone else would find much of it useful. I think I need to work more on it and see if it can be turned into a simple module to attach plugins to, sort of like Addex, but just for mutt configuration.
Dave Rolsky has written up a recap of Frozen Perl from the organizers' point of view. Some interesting tidbits:
The workshop came together beautifully. I was very impressed with what Dave & Co. pulled together.
Alan Morgan wrote to ask if there was a difference between uniquely-decodable (UD) codes for strings and for streams. That is, is there a code for which every finite string is UD, but for which some infinite sequence of symbols has multiple decodings.
I pondered this a bit, and after a little guessing came up with an example: { "a", "ab", "bb" } is UD, because it is a suffix code. But the stream "abbbbbbbbbb..." can be decoded in two ways.
After I found the example, I realized that I shouldn't have needed to guess, because I already knew that you sometimes have to see the last symbol of a string before you can know how to decode it, and in such a code, if there is no such symbol, the decoding must be ambiguous. The code above is UD, but to decode "abbbbbbbbbbbbbbb" you have to count the "b"s to figure out whether the first code word is "a" or "ab".
Let's say that a code is UD+ if it has the property that no two infinite sequences of code words have the same concatenation. Can we characterize the UD+ codes? Clearly, UD+ implies UD, and the example above shows that the converse is not true. A simple argument shows that all prefix codes are UD+. So the question now is, are there UD+ codes that are not prefix codes? I don't know.
[ Addendum 20080303: Gareth McCaughan points out that { "a", "ab" } is UD+ but not prefix. ]
I've noticed a few bugs in Angerwhale these days that are annoying people. For some reason, the CAPTCHA is really hard to guess (it has to be a bug, I'm sure that I'm typing it in correctly), and of course Crypt::OpenPGP is choking on UTF-8 regularly. I found it ironic that I couldn't use my own perl software to post a rant about people misusing the perl UTF-8 flag. (Not signing the post fixed the problem; my code is mostly correct, but the modules I use are broken...)
Anyway, work on Angerwhale has been slow-going because Crypt::OpenPGP doesn't work on 5.10. This means I can't run the test suite, among other things. I need to get rid of that broken module anyway, so I will sometime this week. Then I can start actively fixing the Angerwhale weirdness and push out a new version.
For now, sorry for the brokenness. It will be fixed Real Soon Now.
Back in 2001, I posted a bunch of mini-essays about class in the United States in my newsgroup on SFF Net — six of them in all, though I’d originally planned on seven to nine; I got distracted before I finished.
I’m planning to repost them here, somewhat edited and updated, as much for my own amusement as anything else, but I welcome comments.
I don’t have a hard and fast schedule for when they’ll appear here, but I thought I’d let any readers know they’re coming.
I often see people (including myself) write code like this:
utf8::encode($data_to_print) if utf8::is_utf8($data_to_print);
This results in irritating bugs, because it's incorrect and doesn't even make sense.
Perl can store text as either latin-1 or utf8 internally. For a
string like ほげ
, the internal representation will be utf8,
because you definitely can't represent Japanese in latin-1. However
(and this is where problems crop up), something like ü
(written as
"\xfc"
) will be represented as latin-1, even if you use utf8
.
This is not something that should concern you, but when you check
utf8::is_utf8
, you've just made Perl's internals your problem. You
want to convert to utf8 regardless of the internal representation, but
your code only converts if the string is already utf8. If the string
isn't utf8, you're saying you don't want to convert it? What!?
The key is to drop the if condition. Always encode
to utf8 if you
want utf8!
print {$the_web_browser} Encode::encode('utf8', $some_string);
Please stop checking utf8::is_utf8
. Fuck the internal
representation!
After staying at Wendy and Liz' (and visting a cave (!) in the Dutch mountains (!!)) I stayed a night at Mark & Cleos place.
Their 1.5 years old son Tycho is very cute (but I'm still happy that my kids are already older...). I managed to get most of my slides done while staying there.
On Thursday evening we prepared the venue for the Workshop (including setting up Wendys huge collection of Perl books). Then a aprox 10 attendees had some chinese food at Marks place. Then we went back to the venue, for more preparation, Fluxx and Whisk(e)y.
A tiny bit after midnight I was done with all my slides (even for the talk I only agreed upon a few hours ago, and a lightning talk), so I started working on Acme::ReturnValue. After finishing a first version I want to bed way to late.
The workshop itself was very nice. Jonathans talk on Perl6 Internals was very interesting (and it prompted me to install Perl6 on my laptop - hard to belive, but things like
work!)perl6 -e 'say "Hello World!"'
I used my new wireless presenter thingy to remote-control my computer during my presentation, which was very nice.
Here are my talks (and their slides):
The evening was spend (as usual) with Fluxx, Whisk(e)y and Cake. I very much enjoyed my stay in the Netherlands. Everybody was very hospitable and I had a lod of fun speaking fake dutch
But I also look forward to beeing home again and spend some time with my family...
The difference between a program and a script isn't as subtle as most people think. A script is interpreted, and a program is compiled.
Of course, there's no reason you can't write a compiler that immediately executes the compiled form of a program without writing compilation artifacts to disk, but that's an implementation detail, and precision in technical matters is important.
Though Perl 5, for example, doesn't write out the artifacts of compilation to disk and Java and .Net do, Perl 5 is clearly an interpreter even though it evaluates the compiled form of code in the same way that the JVM and the CLR do. Why? Because it's a scripting language.
Okay, that's a facetious explanation.
The difference between a program and a script is if there's native compilation available in at least one widely-used implementation. Thus Java before the prevalence of even the HotSpot JVM and its JIT was a scripting language and now it's a programming language, except that you can write a C interpreter that doesn't have a JIT and C programs become scripts.
Hm.
Of course, if someone were to write an extra optimizer step for Perl 5 to evaluate certain parts of the optree and generate native code in memory on certain platforms without writing it out to disk (uh oh...) and then execute that code under certain conditions, all Perl 5 scripts would automatically turn into programs.
You know, like .pmc files, or Python's .pyc files. Uh.
As well, if more people use Punie (Perl 1 on Parrot) this year than native Perl 1 -- a possibility -- then Perl 1 scripts automatically become Perl 1 programs becaues Punie can use Parrot's JIT. I don't know if this powerful upgrade from script to program is retroactive, but I see no reason why not.
Perl 5 scripts were briefly programs while Ponie was viable, but the removal of the code from the Parrot tree has now downgraded them back to scripts. We apologize for the inconvenience.
To summarize, if you have a separate compilation step visible to developers, you have programs. If not, you have scripts. An exception is that if you have a separate, partial compilation step at runtime and not visible to users, then you may have programs. The presence of one implementation that performs additional compilationy thingies at runtime instantly upgrades all scripts to programs, while the presence of an interpreter for a language in which people normally write programs, not scripts, does not downgrade programs to scripts. Program-ness is sticky.
I hope this is now clear.
Ironically some JavaScript implementations have JITs, so the colloquial name of the language should change from JavaScript to JavaProgram.
Script bad, four-legs good.
In a triumph of PR right up there with suggesting that Intel executives ever badgered Microsoft executives into doing anything, IBM this week introduced a new generation of mainframe computers. The IBM System z10 is smaller, faster, cooler, has more memory, more storage -- more of everything in fact -- and all that is crammed into less of everything than was the case with the z9 machine it replaces. Touted as more of a super-duper virtualization server than traditional big iron, the only problem with the z10 is that every bit of its superior performance can be easily attributed to Moore's Law. The darned thing actually should be faster than it is. There's a mainframe revolution going on all right, but it's not at IBM. The real mainframe revolt is taking place inside your generic Linux box, as well as at an outfit called Azul Systems.
I'm perfectly happy for IBM to introduce a great new mainframe computer. It's just that the 85 percent faster, 85 percent smaller and a little bit cheaper z10 is coming three years after the z9, and Moore's Law says sister machines that far apart ought to be 200 percent faster, not 85 percent -- a fact that IBM managed to ignore while touting the new machine's unsubstantiated equivalence to 1,500 Intel single-processor servers.
Where were the hard questions? Did anyone do the math? The tricked-out z10 that's the supposed equivalent of 1,500 beige boxes costs close to $20 million, which works out to $13,333 per beige box -- hardly a cost savings. Even taking into account the data center space savings, power savings, and possibly (far from guaranteed) savings on administration, the z10 really isn't much of a deal unless you use it for one thing and one thing only -- replacing a z9.
So the newfangled mainframe is really just an oldfangled mainframe after all, which I am sure is comforting for folks who like to buy oldfangled mainframes.
But those sketchily described IBM benchmarks are, themselves, dubious. IBM never fully explains its own benchmarks nor does it even allow others to benchmark IBM mainframe computers. So nobody really knows how fast the z10 is or how many Intel boxes it can replace if those boxes are actually DOING something.
Remember the stories folks like me wrote a few years back about an earlier IBM mainframe running 40,000+ instances of SUSE Linux under VM on one machine? I wonder how many of those 40,000 parallel Linux images were simultaneously running Doom? My guess is none were.
Far more interesting to me is the vastly increasing utility of Linux as what I would consider a mainframe-equivalent operating system, primarily due to the open source OS's newfound skill with multiple threads that goes a long way toward making efficient use of those multi-core processors we all are so excited to buy yet barely use.
As I wrote a few weeks ago in a column on semiconductor voltage leakage of all things, all this multi-core stuff is really about keeping benchmark performance up while keeping clock speeds down so the CPUs don't overheat. Unlike the benchmark programs, most desktop applications still run on a single processor core and have no good way to take efficient advantage of this extra oomph.
But that's changing. Linux used to be especially bad at dealing with multiple program threads for example -- so bad the rule of thumb was it simply wasn't worth even trying under most conditions. But that was with the archaic Linux 2.4 kernel. Now we have Linux 2.6 and a new library called NPTL or Native POSIX Thread Library to change all that.
NPTL has been in the enterprise versions of Red Hat Linux for a while, but now it is here for the rest of us, too. With NPTL, hundreds of thousands of threads on one machine are now very possible. And where it used to be an issue when many threads competed for data structures (think about 1,000 threads all trying to update a hash table), we now have data structures where no thread waits for any other. In fact, if one thread gets swapped out before it's done doing the update, the next thread detects this and helps finish the job.
The upshot is superior performance IF applications are prepared to take advantage.
"My e-mail application runs on a four-core Opteron server," says a techie friend of mine, "but I've seen it have over 4,000 simultaneous connections - 4,000 separate threads (where I'm using "thread" to describe a lightweight process) competing for those four CPU's. And looking at the stats, my CPUs are running under five percent almost all the time. This stuff really has come a long way."
But not nearly as far as Azul Systems has gone in ITS redefinition of the mainframe -- extending further than any other company, as far as I can tell, models for thread management and process concurrency.
Azul makes custom multi-core server appliances. You can buy a 14u Azul box with up to 768 processor cores and 768 gigabytes of memory. The processors are of Azul's own design, at least for now.
But what's a server appliance? In the case of Azul, the appliance is a kind of Java co-processor that sits on the network providing compute assistance to many different Java applications running on many different machines.
Java has always been a great language for writing big apps that can be virtualized across a bunch of processors or machines. But while Java was flexible and elegant, it wasn't always very fast, the biggest problem being processor delays caused by Java's automatic garbage collection routines. Azul handles garbage collection in hardware rather than in software, making it a continuous process that keeps garbage heap sizes down and performance up.
Language geeks used to sit around arguing about the comparative performance of Java with, say, C or C++ and some (maybe I should actually write "Sun") would claim that Java was just as fast as C++. And it was, for everything except getting work done because of intermittent garbage collection delays. Well now Azul -- not just with its custom hardware but also with its unique multi-core Java Virtual Machine -- has made those arguments moot: Java finally IS as fast as C++.
But for that matter there is no reason to believe that Azul's architecture has to be limited to Java, either, and can't be extended to C++, too.
To me what's exciting here is Azul's redefinition of big iron. That z10 box from IBM, for example, can look to the network like 1,500 little servers running a variety of operating systems. That's useful to a point, but not especially flexible. Azul's appliance doesn't replace servers in this sense of substituting one virtualized instance for what might previously have been a discrete hardware device. Instead, Azul ASSISTS existing servers with their Java processing needs with the result that fewer total servers are required.
Servers aren't replaced, they are made unnecessary at a typical ratio of 10-to-one, according to Azul. So what might have required 100 blade servers can be done FASTER (Azul claims 5-50X) with 10 blade servers and an Azul appliance. Now that Azul box is not cheap, costing close to $1,000 per CPU core, but that's comparable to blade server prices and vastly cheaper than mainframe power that isn't nearly as flexible.
And flexibility is what this is all about, because Azul's assistance is provided both transparently and transiently. Java apps don't have to be rewritten to accept assistance from the Azul appliance. If it is visible on the network, the appliance can assist ANY Java app, with that assistance coming in proportion to the amount of help required based on the number of cores available.
Now imagine how this would work in a data center. Unlike a traditional mainframe that would take over from some number of servers, the Azul box would assist EVERY server in the room as needed, so that you might need a big Azul box for every thousand or so servers, with that total number of servers dramatically diminished because of the dynamically shared overhead.
This is simply more efficient computing -- something we don't often see.
There are other concurrency architectures out there like Appistry (which I wrote about back when it was called Tsunami before we unfortunately HAD a Tsunami -- what sort of marketing bad luck is that?). But where Appistry spreads the compute load concurrently across hundreds or thousands of computers, Azul ASSISTS hundreds or thousands of servers or server images with their compute requirements as needed.
Bear Stearns runs its back office with Azul assistance, but many customers use Azul boxes to accelerate their websites.
Since I am not a big company guy who cares very much about what big companies do, what I see exciting about Azul's approach is how it could be applied in the kinds of data centers where I am typically renting either virtual or dedicated servers. If an Azul box were installed on that network, my little app would instantly and mysteriously run up to 50 times faster.
Cool.
The Register today (29th Feb 2008), has highlighted an all too common issue for several thousand people. Their birthdays fall on the leap day of a leap year. To emphasise their plight they have written a snippet of perl code to show how easy it is to get the number of days in a year. Obviously it might be a little more robust if they use something like DateTime, but the general rule is there. They have now taken on Toys R Us and Microsoft to fix the problem in their websites and software respectively. I wish them luck :)
So, it seems that every time I switch workspace, Spaces helpfully decides that I want Finder as the front most application. How useful.
This makes Spaces a waste of space.
Dear lazyweb, which virtual desktop system is useful?
The Perl 6 design team met by phone on 27 February 2008. Larry, Allison, Patrick, Will, Jerry, Jesse, Nicholas, and chromatic attended.
Larry:
is export
on themmy
to get non-exported versionsgimme
program that translate thatvws
rule for vertical whitespacePatrick:
Jerry:
Patrick:
Allison:
Jerry:
fudge
workinggettext
so we can work toward internationalizing ParrotJesse:
c:
Nicholas:
Jerry:
Allison:
Jerry:
Allison:
Jerry:
Allison:
Jerry:
Allison:
c:
Allison:
So I was chatting to Danny O’Brien a few days ago. He noted that he’d reduced his Spamassassin “this is spam” threshold from the default 5.0 points to 3.7, and was wondering what that meant:
I know what it means in raw technical terms — spamassassin now marks anything >3.7 as spam, as opposed to the default of five. But given the genetic algorithm way that SA calculates the rule scoring, what does lowering the score mean? That I’m more confident that stuff marked ham is stuffed marked ham than the average person? That my bayesian scoring is now really good?
Do people usually do this without harmful side-effects? What does it mean about them if they do it?
Does it make me a good person? Will I smell of ham? These are the things that keep me awake at night.
It’s a good question! Here’s what I responded with — it occurs to me that this is probably quite widely speculated about, so let’s blog it here, too.
As you tweak the threshold, it gets more or less aggressive.
By default, we target a false positive rate of less than 0.1% — that means 1 FP, a ham marked as spam incorrectly, per 1000 ham messages. Last time the scores were generated, we ran our usual accuracy estimation tests, and got a false positive rate of 0.06% (1 in 1667 hams) and a false negative rate of 1.49% (1 in 67 spams) for the default threshold of 5.0 points. That’s assuming you’re using network tests (you should be) and have Bayes training (this is generally the case after running for a few weeks with autolearning on).
If you lower the threshold, then, that trades off the false negatives (reducing them — less spam getting past) in exchange for more false positives (hams getting caught). In those tests, here’s some figures for other thresholds:
SUMMARY for threshold 3.0: False positives: 290 0.43% False negatives: 313 0.26%
SUMMARY for threshold 4.0: False positives: 104 0.15% False negatives: 1084 0.91%
SUMMARY for threshold 4.5: False positives: 68 0.10% False negatives: 1345 1.13%
so you can see FPs rise quite quickly as the threshold drops. At 4.0 points, the nearest to 3.7, 1 in 666 ham messages (0.15%) will be marked incorrectly as spam. That’s nearly 3 times as many FPs as the default setting’s value (0.06%). On the other hand, only 1 in 109 spams will be mis-filed.
Here’s the reports from the last release, with all those figures for different thresholds — should be useful for figuring out the likelihoods!
In fact, let’s get some graphs from that report. Here is a graph of false positives (in orange) vs false negatives (in blue) as the threshold changes…
and, to illustrate the details a little better, zoom in to the area between 0% and 1%…
You can see that the default threshold of 5 isn’t where the FP% and FN% rates meet; instead, it’s got a much lower FP% rate than FN%. This is because we consider FPs to be much more dangerous than missed spams, so we try to avoid them to a higher degree.
An alternative, more standardized way to display this info is as a Receiver Operating Characteristic curve, which is basically a plot of the true positive rate vs false positives, on a scale from 0 to 1.
Here’s the SpamAssassin ROC curve:
More usefully, here’s the ROC curve zoomed in nearer the “perfect accuracy” top-left corner:
Unfortunately, this type of graph isn’t much use for picking a SpamAssassin threshold. GNUplot doesn’t allow individual points to be marked with the value from a certain column, otherwise this would be much more useful, since we’d be able to tell which threshold value corresponds to each point. C’est la vie!
(GNUplot commands to render these graphs are here.)
I’m very pleased to announce the release of SVN::Notify 2.70. You can see an example of its colordiff output here. This is a major release that I’ve spent the last several weeks polishing and tweaking to get just right. There are quite a few changes, but the two most important are imporoved character encoding support and output filtering.
I’ve had a number of bug reports regarding issues with character encodings. Particularly for folks working in Europe and Asia, but really for anyone using multibyte characters in their source code and log messages (and we all do nowadays, don’t we?), it has been difficult to find the proper incantation to get SVN::Notify to convert data from and to their proper encodings. Using a patch from Toshikazu Kinkoh as a starting-point, and with a lot of reading and experimentation, as well as regular and patient tests on Toshikazu’s and Martin Lindhe’s production systems, I think I’ve finally got it nailed down.
Now you can use the --encoding
(formerly --charset
), --svn-encoding
, and --diff-encoding
options—as well as --language
—to get SVN::Notify to do the right thing. As long as your Subversion server’s OS supports an appropriate locale, you should be golden (mine is old, with no UTF-8 locales :\). And if all else fails, you can still set the $LANG
environment variable before executing svnnotify
.
There is actually a fair bit to know about encodings to get it to work properly, but if you use UTF-8 throughout and your OS supports UTF-8 locales, you shouldn’t have to do anything. You might have to set --language
in order to get it to use the proper locale. See the new documentation of the encoding support for all the details. And if you still have problems, please do let me know.
Much sexier is the addition of output filtering in SVN::Notify 2.70. I got pretty tired of getting feature requests for what are essentially formatting modifications, such as this one requesting support for KDE-style keyword support. I myself was using Trac wiki syntax in commit messages on a recent project and wanted to see them converted to HTML for messages output by SVN::Notify::HTML::ColorDiff.
So I finally sat down and gave some though on how to implement a simple plugin architecture for SVN::Notify. When I realized that it was generally just formatting that people wanted, it became simpler: I just needed a way to allow folks to write simple output filters. The solution I came up with was to just use Perl. Output filters are simply subroutines named for the kind of output they filter. They live in perl packages. That’s it.
For example, say that your developers write their commit log messages in Textile, and rather than receive them stuck inside <pre>
tags, you’d like them converted to HTML. It’s simple. Just put this code in a Perl module file:
package SVN::Notify::Filter::Textile; use Text::Textile (); sub log_message { my ($notifier, $lines) = @_; return $lines unless $notify->content_type eq 'text/html'; return [ Text::Textile->new->process( join $/, @$lines ) ]; }
Put the file, SVN/Notify/Filter/Textile.pm somewhere in a Perl library directory. Then use the new --filter
option to svnnotify
to put it to work:
svnnotify -p "$1" -r "$2" --handler HTML::ColorDiff --filter Textile
Yep, that’s it! SVN::Notify will find the filter module, load it, register its filtering subroutine, and then call it at the appropriate time. Of course, there are a lot of things you can filter; consult the complete documentation for all of the details. But hopefully this gives you a flavor for how easy it is to write new filters for SVN::Notify. I’m hoping that all those folks who want featurs can now stop bugging me and writing their own filters to do the job, and uploading them to CPAN for all to share!
To get things started, I scratched my own itch, writing a Trac filter myself. The filter is almost as simple as the Textile example above, but I also spent quite a bit of time tweaking the CSS so that most of the Trac-generated HTML looks good. You can see an example right here. Thanks to a number of bug fixes in Text::Trac, as well as Trac-specific CSS added via a filter on CSS output, it works beautifully. If I’m feeling motivated in the next week or so, I’ll create a separate CPAN distribution with just a Markdown filter and upload it. That will create a nice distriution example for folks to copy to creat their own. Or maybe someone on the Lazy Web Will do it for me! Maybe you?
I wish I’d thought to do this from the beginning; it would have saved me from having to add so many features/cruft to SVN::Notify over the years. Here’s a quick list of the features that likely could have been implemented via filters instead of added to the core:
--user-domain
: Combine the SVN username with a domain for the Fromheader.
--add-header
: Add a header to the message.--reply-to
: Add a specific header to the message.--subject-prefix:
: Modify the message subject.--subject-cx
: Add the commit context to the subject.--strip-cx-regex
: More subject context modification.--no-first-line
: Another subject filter.--max-sub-length
: Yet another!--max-diff-length
: A filter could truncate the diff, although this might be tricky with the HTML formatting.--author-url
: Modify the metadata section to add a link to the author URL.--revision-url
: Ditto for the revision URL.--ticket-map
: Filter the log message for various ticketing system strings to convert to URLs. This also encompasses the old --rt-url
, --bugzilla-url
, --gnats-url
, and --jira-url
options.--header
: Filter the beginning of the message.--footer
: Filter the end of the message.--linkize
: Filter the log message to convert URLs to links for HTML messages.--css-url
: Filter the CSS to modify it, or filter the start of the HTML to add a link to an external CSS URL.--wrap-log
: Reformat the log message for HTML.Yes, really! That’s about half the functionality right there. I’m glad that I won’t have to add any more like that; filters are a much better way to go.
So download it, install it, write some filters, get your multibyte characters output properly, and enjoy! And as usual, send me your bug reports, but implement your own improvements using filters!
Read more of this story at Slashdot.
The UKUUG Spring 2008 Conference will be held at the Birmingham Conservatoire (Birmingham, UK) from 31st March to 2nd April 2008. Monday features 3 full day tutorials, with Tuesday and Wednesday taken up by shorter technical presentations. You can see the current schedule here.
Birmingham Perl Mongers are helping to coordinate the event, with several Perl Mongers from around the UK offering to present talks. Dave Cross will be holding a one-day Perl tutorial of his Teach-In session, with Matt S Trout, Tom Hukins, Mike Whitaker, Daniel Giribet, Barnabas Aspray, Mark Gledhill, Robin Doran, Brian McCauley, Jon Allen and me presenting technical talks discussing Perl.
The event is a UKUUG event and there are discounted rates for UKUUG members. You can attend just a tutorial, just the technical talk days or the whole 3 days. See Payment Details for all the options and prices. For those travelling from further afield who would like to know of some local hotels, the Accommodation page lists several nearby hotels. The conference venue is situated at the west end of New Street and is within 5-10 minutes walk of several hotels, including all those available during YAPC::Europe in 2006. Both GUADEC and PyConUK were held there in 2007, and proved a very suitable venue.
The Early Bird rates for the event are due to end on March 10th, so if you're thinking of coming along, please book soon to take advantage of the reduced rates.
To facilitate the triage we’re going to be doing this week for Fx3/1.9 we’ll be making some changes to various flags. There’s a few goals here:
* Identifying the set of bugs that will absolutely block the release of Firefox 3
* From those bugs, identify the set of bugs that we need to ship in a beta to get wider testing before ship.
* Ensure that bugs we won’t get done for this release don’t get lost in the shuffle.
In order to do this, we are going to work against the following plan, starting Real Soon Now:
* The current blocking1.9 flag will be renamed to tracking1.9 on a temporary basis.
* A new blocking1.9 flag will be created, along with a wanted-next flag to track bugs that won’t make 1.9.0 but should be tracked.
* All b4 blockers will move over to the new blocking flag immediately.
* All outstanding nominations against the old blocking flag will be migrated as well.
* Drivers will triage all of the open bugs marked as tracking1.9+, moving flags to one or more of blocking1.9, wanted-next, or wanted1.9.0.x as appropriate. the tracking1.9 flag will be cleared as we go.
* When there are no bugs with the tracking1.9 flag left, we’ll obsolete the flag and rename it to blocking1.9 so historical queries continue to work.
For developers and triagers:
Please don’t be alarmed if your P1/P2 bugs no longer appear as blockers, we’ll be retriaging aggressively and plan to be complete this week. If you have opinions on your bugs, please add comments making your recommendations for drivers.
The new blocking1.9 should be used like the old one, both for nominating blockers for beta 4 and for final. If a bug was previously marked as a blocker but hasn’t been retriaged, please nominate again to expedite the process.
For questions or feedback, please see the matching post in dev.planning.
The folks over at PlainBlack were nice enough to offer up some graphic design assistance and created a set of banners for people to help promote YAPC::NA 2008. Please feel free to use these banners and li nke to this site to help spread the word about YAPC.
As part of Mozilla Corporation’s ongoing stability and security update process, Thunderbird 2.0.0.12 is now available for Windows, Mac, and Linux as a free download from www.getthunderbird.com.
Due to the security fixes, we strongly recommend that all Thunderbird users upgrade to this latest release.
If you already have Thunderbird 2.0.0.x, you will receive an automated update notification within 24 to 48 hours. This update can also be applied manually by selecting “Check for Updates…” from the Help menu.
For a list of changes and more information, please review the Thunderbird 2.0.0.12 Release Notes.
Please note: If you’re still using Thunderbird 1.5.0.x, this version is no longer supported and contains known security vulnerabilities. Please upgrade to Thunderbird 2 by downloading Thunderbird 2.0.0.12 from www.getthunderbird.com.
At today’s Firefox 3 / Gecko 1.9 meeting we reviewed the remaining number of P1 and P2 blockers on our lists, and decided that we would go ahead with tonight’s Firefox 3 Beta 4 code freeze tonight at 11:59pm PST and then bake nightly builds for a few days before handing off the code to the Build and Quality Assurance teams.
During the baking period, all Firefox 3/ Gecko 1.9 component owners have been asked to triage their blocker nomination and blocking bug lists by Friday noon PST in order to determine:
Mike Connor has posted criteria to be used when assigning priorities to blocking bugs. If possible, bug owners should add an estimate of the time required and precieved difficulty of fixing a bug.
Tree management after tonight’s freeze
After tonight’s freeze we’ll be using the same process we used for the last beta. The tree will be closed to bake, and during that time we would appreciate your help in regression spotting and fixing. The only bugs that can land during this baking period are those marked with the “approval1.9b4+” flag.
Please note that if you believe a bug should block the release of Firefox 3 Beta 4, please make sure it is either marked or nominated as a P1 blocker and set the target milestone to “Firefox 3 beta 4″ or “mozilla1.9beta4″. We are currently frozen for localization work, so no new strings will be accepted at this time. You may wish to mark bugs with the following keywords:
After this Friday, we will have more information about the delivery schedule for this beta. Look for announcements here and on the mozilla.dev.planning group.
Big problem: which queue to choose? Having learnt about Queuing Theory at school, and knowing that single queue-multiple consumer is as efficient as it gets, it's always an agonizing decision. You have to balance a number of factors: how many people in the queue, how many items do people want to buy, do they look impatient or vagued out. It's better to be behind a person buying 10 of one thing, than one item of ten different things. And, although it pains me to admit it, old people are Bad News. Statistically, you'll waste 30 seconds to five minutes with them in front of you.
So I was pondering this the other day, and reflecting on my worldview, how I tend to algorithmatise things (figuring out the shortest path between the four shops I need to go to, establishing contigency plans in case a shop doesn't have a product I need, that sort of stuff. When house-cleaning I try to keep my hands full. I hate going into room A to fetch something for room B, and realising there was something in room B which should be tidied away in room A). Geeking out, basically.
And as I stood there ruminating on this, I was idly watching the woman at the front of the line, and became more and more alarmed. She was just stacking up the products that the cashier swiped...but not putting them into bags. I couldn't figure out what she was doing, but figured it was bad news for me. And on and on it went, until the cashier had checked out every last item. Then she handed over her credit card, had it swiped, punched in her pin, and only once she had received her docket did she begin to bag up the stuff she'd bought. And I thought "You f5cking moron: can't you deal with asynchronous events?" Because the cashier couldn't start checking out the items of the next person until she'd cleared the decks. Which took a non-trivial amount of time, during which no other work could be done.
Meantime I note that the people who were at the ends of the other queues are now close to being served, and I've still got two people in front of me. But I'm in too deep at this point to bear the cost of cutting my losses and requeuing on another line.
The next person went through without a hitch: a few items, had cash ready, out straight away.
Then the person in front of me went through. Seemed to be okay: started bagging up the goods as soon as they came through, but she had a fair number of things to buy. Then it turns out that she has a Method. All the fruit goes together, the stuff that requires Refrigeration goes together, tins kept apart from packets and so on. And the cashier is done quickly and announces the price. But does she do anything? Noooooooooo! She just keeps putting stuff in bags. And the cashier, who is polite (the Customer is always Right and all that) says nothing.
At this point I just take a deep breath and resign myself. The people who were at the ends of the other queues are all long since gone. At times like this I can't help thinking that a lot of what we learn in Computer Science is applicable to the Real World. And one of the cardinal sins is stalling the pipeline. When you have people queued up to do something, you want to get in and get out as fast as possible, just like an instruction on a CPU. Because wait states are just time lost. Gone, never to come back.
I really, really hate when that happens.
So, I have a database with a particularly nasty design decision. We have people who belong to cost centres, usually only one, but sometimes with a prorata on two cost centres. The programmer responsable for creating the table denormalised things, so rather than having
EMP1 UNIT1 50%
EMP1 UNIT2 50%
EMP2 UNIT3 100%
EMP3 UNIT1 100%
we have something that looks like
EMP1 UNIT1 UNIT2 50 50
EMP2 UNIT3 null 100 0
EMP3 UNIT1 null 100 0
That is, both cost centre ids in the same table, the second one usually null. It is a given that there will never be more than two. This table is of course an utter bitch to work with. Turns out we can cheat a bit, by only keeping track of the rate of the first centre, the second is just 100-first (which also helps cut down round-offs). Let us create a table to play with:
create table t1 (
id_person varchar(10),
rate number(5,2),
unit1 varchar(3),
unit2 varchar(3),
val1 number(10),
val2 number(10)
);
insert into t1 values ('alice', 1, 'U1', null, 10, 20);
insert into t1 values ('bob', 1, 'U2', null, 4, 8);
insert into t1 values ('carol', 0.5, 'U1', 'U2', 300, 600);
insert into t1 values ('david', 0.2, 'U1', 'U3', 6000, 8000);
Now I want the sum the values val1 and val2 by unit, keeping in mind that for 'david', VAL1 6000 * 0.2 = 1200 is summed to U1, and the difference, 4800, to U3. Similarly, for VAL2, 1600 to U1 and 6400 to U3. In other words, I want the following result set:
U1 1360 1920
U2 154 308
U3 4800 6400
Now the only way that I can see is to:
This gives the following:
select
S.UNIT UNIT
,sum(V1) V1_TOT
,sum(V2) V2_TOT
from (
select
unit1 UNIT
,sum(val1 * rate) V1
,sum(val2 * rate) V2
from
t1
group by
unit1
union select
unit2 UNIT
,sum(val1 - val1 * rate) V1
,sum(val2 - val2 * rate) V2
from
t1
where
unit2 is not null
group by
unit2
) S
group by S.UNIT
order by S.UNIT
That's pretty ugly. Is there a better way?
/me is annoyed
I just spent a while chasing a stupid bug. I have an big log directory that I have to clean up. The number of files caused the shell's wildcard expansion to fail. So I wrote some perl to move things around.
And it didn't do anything. To cut a long story short, I needed the total number of files in the directory. So I had something along the lines of:
perl -le 'print scalar(glob("*"))'
That does not return the number of files. It returns the name of the first file. Or the last. I don't care.
You have to force array context and throw it away:
perl -le 'print scalar(()=glob("*"))'
This prints 31192. I wish Perl didn't do this. Nice interview question though, I guess.
After the nice 0100000 party on Thursday (and a nice loot) I took the sleeper train to Köln on Friday night. By some very lucky coincidence I got a 4-person compartment on my own - yay!
The next day I walked a bit through the central area of Köln (around the Cathredral). Quite nice for a german city... I discovered a space invader and a fake medevial castle on a boat (some kind of museum).
In Brussels, I went straigth to FOSDEM where I met a few Perl people (Mark, Juerd, Leon, ..) at the Perl 6 talk. I spend most of the day talking with them. In the evening we had yet another birthday "party" (together with Leon) - and to my big surprise I discovered that I actually like Cherry Beer (because it doesn't taste like beer I guess)
Sunday night I drove with Wendy & Liz to their place, where I'm staying now. It's very nice and they are very hospitable. Yesterday Jonathan arrived from Kiev, and we did a small bicycle tour around the neighbourhood. I'm switching between working, preparing slides, chatting & playing Fluxx. Very nice...
... what I'm seeing in Ruby is that the many ways have been transformed into idioms and guidelines. There are no hard rules, but the community have evolutionary evolved idioms that work and found out many of the ways that doesn't work.
Ruby has almost all of the flexibility of Perl to do things in different ways. But at the end of the day, none of the Perl problems tend to show up. Why is this?
— Ola Bini, Language design philosophy: more than one way?
Why? Probably because you're not a very good Perl programmer.
A better question is "Why do programmers experienced with a language so often fail to have the maintainability problems that plague novices?" That might have led nicely into a discussion of idioms and the evolution of good style in a language which encourages such things.
See also the discussion why Java fans luuurve their IDEs and can't understand why those of us who can go through the TDD cycle once or twice before Eclipse finishes loading the whole project think that Java's syntax might be a touch verbose. Another fun discussion is "Why don't you Lunix types make one-click installers you can just download and click to install?"
I suppose only the familiar is awesome, and the unfamiliar is stupid and scary.
Greg Costikyan has been saying this for a decade or more, but this is well-written at least:
The truth is that, for the most part, we don’t have anything like game criticism, and we need it—to inform gamers, to hold developers to task, and to inform our broader cultural understanding of games and their importance and impact on our culture.
We need our own Pauline Kaels and John Simons—and we need to ensure that when they appear, no one insists that they attach a damn numerical score to their writing, because that is wholly irrelevant to the undertaking of writing seriously about games.
Event: Technical Meeting
Date: Wednesday 27th February 2008
Times: from 7pm onwards (see below)
Venue: The Victoria, 48 John Bright Street, Birmingham, B1 1BN.
Details: http://birmingham.pm.org/cgi-bin/brum.pl?act=tech-next
Details
This month we're featuring a special guest night with Tony Cooper coming along to give us two talks. The first looks at the VCS Monotone, and the second looks at creating GUI tools with Perl, Glade and Gtk2.
Once again we'll be at The Victoria this month. The pub is on the corner of John Bright Street and Beak Street, between the old entrance to the Alexandra Theatre and the backstage entrance. If in doubt, the main entrance to the Theatre is on the inner ring road, near the Pagoda roundabout. The pub is on the road immediately behind the main entrance. See the map link on the website if you're stuck.
As always entry is free, with no knowledge of Perl required. We'd be delighted to have you along, so feel free to invite family, friends and colleagues ;)
We'll be in the venue from about 6.30pm, and have booked The Boardroom for the night. Order food as you get there, and we'll aim to begin talks at about 8pm. I expect talks to finish about 9.30pm, with plenty of time for discussion and planning for meetups over the weekend in London.
Venue & Directions:
The Victoria, 48 John Bright Street, Birmingham, B1 1BN (Tel: 0121 633 9439)
- Pub Details
- Picture
- Google Map
The venue is approximately 5-10 minutes walk from New Street station, and about the same from the city centre. On street car parking is available see full details and directions on the website.
Times:
These are the rough times for the evening:
Please note that beer will be consumed during all the above sessions ;)
Rob Griffiths IM'ed me saturday while I was sitting at ATL airport, asking what my day rates were.
My normal reply was "depends on the task", and asked him what he had in mind. As he described it, it fleshed out to all of 10 lines in my head, so I said "well, buy me lunch and we'll call it even". He couldn't believe that it could be that short to create or to write.
Ten minutes later, I showed him the code that he added to a hint for today about renaming old iChat logs.
Hadn't tried it on my own data, but told him "it should probably work". I'm pretty good at previewing code in my head. :) He tried it, and it worked fine for his data. Yeay.
So there I am, famous again. And got a free lunch for 10 minutes work. Not bad.
Flickr sure has some passionate users!
Forget Markup Barbie… I want Unicode Barbie. When you pull her string, she says “text is hard.”
The Home Office said a mandatory database "would raise significant practical and ethical issues".
Home Office minister Tony McNulty told BBC that a national database was not a "silver bullet" and that it would raise practical as well as civil liberties issues.
"How to maintain the security of a database with 4.5m people on it is one thing," he said.
"Doing that for 60m people is another."
Politicians listening to reason and experts and not going to do something (that can be spun as doing something useful) because it's impractical? Who are you and what have you done with the real politicians?
I wrote a review of "Catalyst" by Jonathan Rockway. You can read it (the review, not the book) on my web site.
Executive summary: I didn't like it much.
We regularly get inquiries about the perl.org, perlmonks.org etc domains like the following:
We are interested in purchasing your domain name. If interested, please let me know what it takes to buy it.If agreeable, we can finalize the deal through escrow.com Monday.
Our typical response:
Hi $spammer,One billion USD. Non-sequential twenty dollar bills preferred, but not required.
You pay the escrow.com fees.
Thanks,- ask
p.s. no, it's not really for sale.
If you have ever seen my show Triumph of the Nerds well then you've also seen my car, a 1966 Ford Thunderbird convertible very similar to the car used in the movie Thelma & Louise (I play the role of Thelma). It is in almost every way a fabulous car. It's going up in value, for one thing. It looks cool. It goes like hell with its 428 cubic-inch V-8 engine. It is heavier than anything else on the road, so in a collision with anything less than a dump truck I win. And thanks to analog technology that handily predates the Clean Air Act, it somehow manages to do all this while getting 22 miles per gallon on the highway, 16 in town. Everything is good about my T-Bird, in fact, except for its wires. My car has bad wires. And shortly YOUR car will have bad wires, too, as will everything else you own that has soldered electrical connections. Everything. Prepare to share my pain.
My T-Bird was built at the absolute apex of 20th century electromechanical automotive technology. A convertible, it has a fully automatic electric top that relies on eight electrical relays firing in sequence to put the top up or down. Here's the typical (and inevitable) failure mode. I'm at the beach with the top down. It starts to rain. Quickly I run to the car, start it, and hit the button to raise the top. First the suicide trunk lid opens until it is fully vertical. Next the electric tonneau cover opens until it is vertical, too. Then the canvas top begins to retract, raising until it, too, is vertical, at which point everything grinds to a halt and I drive my car home in the pouring rain at five miles per hour, traffic honking behind me, and all three broken parts sticking eight feet into the sky.
Victim to a succession of good and bad mechanics, some well-intentioned but all in it for the money, I have spent thousands of dollars over the years replacing electrical parts -- window motors, switches, relays for both the top and the sequential tail lights -- all to little avail. New parts failed as quickly as old parts. Eventually I abandoned all hope of viewing my car as a restoration and replaced the relays with brilliant little computers from British Columbia. Now the hydraulics worked beautifully but all it really meant was that the top no longer failed in mid-sequence: it would either work fine or not at all.
When you've replaced everything else the problem has to be with what's left, which in the case of my car was the wires, themselves. Over the years the wires had somehow corroded inside their insulation and the terminals had lost their mojo. I had been replacing perfectly good switches and motors (and knowledgeable folks had been selling me switches and motors) that would have been helped more by simply replacing the terminals or, better still, the wires. Some experts think Ford just got a bad batch of wire back in 1966 -- that this problem is isolated -- but I don't care. So what if my car is two years older than my wife? All her parts seem to be working just fine, why shouldn't my T-Bird?
Which brings me to you, or rather to all of your soldered devices that are two years old or less. Most of these are now assembled using solder joints that have no lead in an effort to save our groundwater and our health. The fact that the lead has been generally replaced with silver or bismuth, both of which are actually greater health risks than lead, well we'll leave that one for Ralph Nader if he decides not to run for President. The longer-term trend is toward all-tin connections, anyway, but they don't work very well, either.
I wrote a column about this back in 2004 (it's in this week's links) that was heavy on information and therefore low on readership. Everything in that column has come to pass and more. Where's my Pulitzer Prize?
Costs have gone up, mean time between failures (MTBF) has gone down (accelerated MTBF tests, which are the only MTBF tests we do anymore, don't reliably pick this up, by the way), and reliability has suffered. Since we don't fix things anymore, it’s hard to say whether your gizmo failed because of bad solder or not, but the problem is becoming worse as a greater percentage of total circuits in use have lead-free solder. The military was especially concerned, even before the whisker crisis.
We're talking about tin whiskers, single crystals that mysteriously grow from pure tin joints but not generally from tin-lead solder joints. Nobody knows how or why these whiskers grow and nobody knows how to stop them, except through the use of lead solder. Whiskers can start growing in a decade or a year or a day after manufacture. They can grow at up to nine millimeters per year. They grow in any atmosphere including a pure vacuum. They grow in any humidity condition. They just grow. And when they get long enough they either touch another joint, shorting out one or more connections, or they vaporize in a flash, creating a little plasma cloud that can carry for an instant hundreds of amps and literally blow your device to pieces.
Since 2006 we have been exclusively manufacturing soldered connections thousands of times more likely to create tin whiskers than previous generation joints made with tin-lead solder. Because of the universal phase-in of the new solder technology and the fact that the solder technologies can't reliably be mixed (old solders mess with new solder joints in the same device through simple outgassing) this means that it is practically impossible to use older, more reliable technology just for mission-critical (even life-critical) connections. So we're all in this tin boat together.
Some experts confidently say that the disparity of joint reliability we are seeing today will go away and that the new joints will become as reliable or even more reliable than the old tin-lead joints as we gain experience with the new processes. What's disturbing, though, is that these experts don't actually know how this increased reliability is likely to be achieved. Just like extrapolating a Moore's Law curve to figure out how fast or how cheap technology is likely to be a decade from now, they have no idea how these gains will be made, just confidence that they will be.
What if the experts are wrong?
Tin whiskers can take out your iPod or your network. They can stop your car cold. They can take down an entire airport or Citibank. They are much more common than most people -- even most experts -- think. The reason for this is that most tin whiskers can't even be seen.
"Maybe it is worth adding," said one expert who prefers to remain anonymous, "that whisker diameters range from 0.1 um to 10 um, while the diameter of a human hair is 70 um to 100 um --- so the largest whisker is only some 15 percent of the diameter of a thin hair, and most are less than 5 percent. A good fraction (of these are) so thin that light waves just pass them by, scattering a bit but not reflecting. So the optical microscope images that (typically used to illustrate whiskers) show only a small fraction of what is really there. Scanning electron microscope (SEM) images are a bit better, but only show a small zone of the sample; also, not many folks are able to acquire SEM images of their equipment. So all too many folks have the idea that whiskers are something that happens to someone else, but never to them. This is an expensive misconception."
What I wonder is whether a cost-benefit analysis of this solder technology changeover was ever done? I haven't seen one.
And if you think this problem is minor, I have been told that just the cost of changing to lead-free solder stands right now at $280 BILLION and climbing. That cost is borne by all of us.
Maybe dumping lead solder was absolutely the right thing to do. Maybe it was absolutely the wrong thing to do. The truth is we haven't the slightest idea the answer to that question and anyone who claims to know is wrong. We didn't know what would happen when we started this and we don't know what we'll get out of it, either, or whether it will be worth the cost. All we know for sure is that a bumpy ride lies ahead.
Fortunately I have new shock absorbers (and a new wiring harness) on my T-Bird.
I understand that there is an inherent and pervasive bias in pure-text communication which makes statements intended to be good-humoured sound sophomoric, makes statements which were intended to be friendly sound smarmy, makes statements which were intended to be enthusiastic sound brash, makes statements intended to be helpful sound condescending, makes statements which were intended to be precise and accurate sound brusque and pedantic, makes statements which were intended to be positive sound neutral, and makes statements which were intended to be neutral seem downright hostile.
From List::Maker’s Changes
file:
0.0.4 Sat Feb 9 11:08:51 2008
* Made globbing magical only in those source files that explicitly load
the module (thanks Steffen)
W00t! Now I can actually use the module instead of making sad eyes and letting out a wistful sigh when I think of it.
Many people are dabbling in functional programming these days. Haskell is more popular than it ever has been, OCaml has it's adherents, and we are starting to see common functional idioms spread throughout the industry.
Why am I not happy?
Here's why. I think that most functional programming languages are fundamentally broken with respect to the software lifecycle. I realize that's a bold statement. Let me back it up.
Imagine these lines of code in any object-oriented language (I'll use Java here because it is familiar to most people, even if they dislike it):
public class X {
public void method() {
...
badMethod();
...
}
...
}
Class X has a method named badMethod
. The method isn't bad because it does something awful while the system is running; it's bad because it does something painful during testing. It could be anything. The method could make a call to our production database and update it, or it could communicate with another system across a socket. It could even touch some low level hardware that causes a mechanical machine arm to do something that it shouldn't be doing over and over again while we are testing.
In an ideal world, people would design their systems so that classes and clusters of classes could be tested independently; the industry is slowly rediscovering the utility of unit testing. But, stuff happens. People make mistakes. They think that they've got it all under control until they discover that they have a big wad of code that they can't easily change and don't understand. At that point, they wish that they could write little tests that help them understand what the hell their code does before and after they change it.
The nice thing, so far, is that most of the object-oriented languages out there leave you an “out.” You can use the features that were built in to provide flexibility to give yourself enough maneuverability to test. In short, we can do this:
public class TestableX extends X {
void badMethod() {
// do nothing
}
}
We can override functionality and build ourselves a wedge that makes testing easy.
The fact that badMethod
is a non-final, override-able method makes it something I call a seam. A seam is a place where you can substitute one piece of functionality for another without editing. Most languages provide seams. The C programming language is rich with them. You can use the preprocessor, function pointers, and link substitution to replace awkward functionality during testing, but it isn't quite as easy as it is in OO.
Virtual function calls in OO languages are a common seam. Java is easy. C++ is a bit more awkward. Ruby and Python are a breeze. The power of these seams is in the fact that you can get between around around the pieces of your application when testing. It isn't just a big hard-coded mass.
What about Haskell, OCaml, and Erlang?
Yes, you can provide alternative modules to link against, but it's clunky. Sure, you can use virtual in OCaml, but who really organizes all of their OCaml code in classes?
Haskell is a bit nicer, most of the code that you'd ever want to avoid in a test can be sequestered in a monad, and I expect that half of the Haskellers reading this will argue that functional purity obviates any need to unit test; it can all be done in QuickCheck. I hear you, but there's distance between here and there and that distance can be a quagmire.
No, the fact is, the late-binding that OO languages provide makes code written in them more easily recoverable. And, this is important because entropy happens. Count on it.
Dear lazyweb,
Given that genuine VT102 terminals are not that practical to lug around, what is the "best" terminal emulator for OS X? I realise that as all are made of software, all will be hateful, but I was hoping that some might be less hateful than others.
to take the utf-8 flag off and make sure all data is still in utf-8 by avoiding any possible latin-1-utf8 auto upgrades.Encode::_utf8_off($stuff)
to assume that "Unicode strings should have UTF-8 flagged, and those without the flag are assumed UTF-8 bytes."utf8::encode($_) if utf8::is_utf8($_)
I've just submitted this talk to YAPC::Asia and will submit it to YAPC::Europe:
We are from the internet - we know the value of open source. Hardware and storage is unfortunately real, but you can outsource it all. This talk will guide you through how to exploit cloud computing today to make you happier and more efficient.
Today, at 4:11pm Pacific Standard Time, an era (or at least 9 years) ended. The cpan-testers mailing list, recipient of a all CPAN tests submitted for CPAN testers processing, no longer resends all of those emails to its 50+ subscribers. The last message to be distributed via email should have been #1049514. Further messages will be kept in the archive, http://www.nntp.perl.org/group/perl.cpan.testers (also home to RSS and Atom feeds). Eventually, we're going to transfer to a proper database backed system that doesn't use email at all.
Several people noted that they used cpan-testers to receive notification of new uploads to CPAN. We've created the cpan-uploads mailing list to receive these notifications. Archive/RSS/Atom is at http://www.nntp.perl.org/group/perl.cpan.uploads. You can subscribe by sending a note to cpan-uploads-subscribe at perl.org.
It's hard to remember to do simple code inspections--the oldest quality technique in the software field--amid the welter of graphical tools, measurement tools, and other fancy tricks the industry has thought up. Code inspections make sense. If you find a flaw during a program run and debug it, you've fixed just one bug. Only a code inspection can fix the potentially hundreds of other similar bugs.
But code inspections are a headache, because logic and consistency are hard to determine. Miska Hiltunen is trying to make them more agreeable by rigidly limiting what you have to look for, and assuring developers that they can get through each inspection in just about an hour. His Tick-the-Code site lists extremely simple things that pop out of poorly constructed code right away, and that supposedly warn you of the risk of deeper problems. Things to look for include unchecked parameters, deeply nested structures, duplicate code, unnecessary comments, unfinished code, and poorly named variables.
How is this relevant to Beautiful Code? A chapter in that book by Laura Wingerd and Christopher Seiwald of Perforce focuses our attention on just such superficial aspects of source code. Simply lining up the code for different cases to show their similarities and differences, these authors claim, can prevent associated bugs. They show how a violation at one stage of one of their principles--deep nesting of control structures, which they, like Hiltunen, try to avoid--coincided with a sharp uptick in errors.
Will Hiltunen's list of simple rules help programmers avoid real bugs? Or is it a focus on trivia that distracts programmers from dealing with real problems in their design and algorithms? Testing and time will tell.
Perhaps we can look at the endeavor this way: design and algorithms are certainly the most important drivers of quality and good performance. But design mostly takes place outside of and before coding, while algorithms are coded by experts who provide them to the rest of us in packages. Most of us who code are cobbling together small programs with lots of glue from standard libraries. Checking for trivial coding inconsistencies may uncover a lot of the problems we need to find.
I found an odd bug in Rails today. Odd, in the sense that it's not so much broken, as working in a way that's different than one would expect.
In a controller, one uses respond_to to present the appropriate response, as determined by the requested mimetype. If you want to respond to a whole clump of mimetypes in the same way, you might expect format.any to act as a catch-all for all the types you didn't already address.
So, for example, if we're writing an authentication system, we want web browsers redirected to the login page. But for non-browsers (eg, curl asking for XML, or Ajax asking for JSON, or some API asking for a vcard, whatever), it doesn't make any sense to direct them to a login page. We should ask them to use http basic authentication.
Here's the way this is handled by the current version of restful_authentication:
def access_denied
respond_to do |format|
format.html do
store_location
redirect_to new_<%= controller_singular_name %>_path
end
format.any do
request_http_basic_authentication 'Web Password'
end
end
end
Except, this doesn't actually work. If you make a request for a non-html mimetype, you'll get a 406 ("that format doesn't work here anymore"), not a 401 ("unauthorized").
Presently, format.any actually looks something like this:
def any(*args, &block)
args.each { |type| send(type, &block) }
end
And, if you dig around a little in the mime_responds tests, you'll see that you're supposed to call #any like so:
respond_to do |type|
type.html { render :text => "HTML" }
type.any(:js, :xml) { render :text => "Either JS or XML" }
end
Which, effectively, makes #any useless as a high-level catch-all for things like #access_denied. It's interesting that you can use it as a sort of multi-type, but that hardly seems the most common application. (Given the scarce documentation and meta-magic shenanigans in MimeType::Responders, I kinda doubt #any is used all that often. I only know about it because of restful_authentication.)
I've created a patch that changes #any's signature. It can now take 0 or any number of arguments. In the case where it has zero arguments, it assumes the mimetype requested (via the Rails-standard format parameter) or the first allowed mimetype (set by the request headers). In other words, it allows code like #access_denied, above from restful_authentication, to work as intended.
If you can, please take a moment to +1 the patch.
… err, I mean, has a weblog.
Welcome aboard, Roy.
Once in a while I hear people talking about using thousands or tens of thousands of tables in one MySQL database and often how it “doesn’t work”. There are two things to say to them:
I’ll elaborate a little on both …
In most cases when extraordinarily many tables are brought up the “correct answer” is: Fix your schema to not duplicate the same table layout for each customer/user/site/…!
Once in a while though there are good reasons to have way too many lots of tables. Even “takes too long to fix the application now” can be a good enough answer sometimes.
My use case was to use the many tables as an extra “index” for a situation that a regular index couldn’t cover. Tens of thousands of tables, here we come.
… which is easy!
In your startup script (/etc/init.d/mysql with the MySQL RPMs), add “ulimit -n 30000” somewhere near the top to allow mysqld to have 30000 open files (and sockets).
In your mysql configuration, add something like the following - adjust the number as appropriate.
set-variable = table_cache=12000
This will let MySQL keep up to 12000 tables open. The default limit is much too low and the system will spend all its time closing and opening the files. The other few thousand handles are free for database connections or whatever. You surely can tune the numbers more, but I haven’t needed to be more specific yet. MySQL uses two filehandles per open table (for MyISAM, it depends on the table type…)
One curious thing you’ll run into is that MySQL can take forever (read: hours!) flushing thousands of tables that have been changed if you do it with a simple “flush tables” or when you are shutting down mysql. That’s of course not “hmn, how curious” but rather insanely frustrating if you were going for a quick restart. This occasionally seems to happen with InnoDB tables too, but in particular with our large MyISAM system (we use fulltext indexes, hence MyISAM) this is a big issue.
With MyISAM tables, you also have a lot of cleaning up to do if the system crashes one way or another with thousands of tables open.
There’s an easy-ish solution to both these problems, though! Just flush the tables individually before the shutdown command and on a regular basis to mitigate the issue if it crashes. Remember the system or MySQL can crash for all sorts of reasons. Recently we had a motherboard with a BIOS or hardware bug that made it unstable after adding more than 16GB memory. On another box, one of the memory sticks went bad so suddenly it came up with 4GB memory less than previously. With MySQL carefully tuned to use all the memory (it was an InnoDB-only installation) it’d try using all 24GB memory and get killed by the kernel when it got above 20! Yeah, that one took some head scratching before I figured it out.
Below is a small program that’ll go through the database twice flushing all tables in all databases and then end with a regular “flush all”. We go through the database twice just in case the first flush took so long that a lot of tables got opened again. Two works for me, depending on your needs you might make it an option. Or a fancy version would check how many open tables are at the end of the run and go through it again if too many are open (and abort with an error if they open faster than they can get closed!). The final “flush tables” worked for me in getting everything closed just before the script exits (and the shutdown or whatever starts).
#!/usr/bin/perl
use strict;
use warnings;
use Getopt::Long qw(GetOptions);
use DBI;
my %args = (
verbose => 0,
user => $ENV{USER},
hostname => '127.0.0.1',
port => 3306
);
GetOptions(\%args,
'verbose!',
'user=s',
'password=s',
'hostname=s',
'port=i',
'database=s@'
);
my $dbh = DBI->connect("dbi:mysql:hostname=$args{hostname};port=$args{port}",
$args{user}, $args{password})
or die DBI->errstr;
my @dbs = $dbh->func('_ListDBs');
for my $db (@dbs) {
next if uc($db) eq 'INFORMATION_SCHEMA';
$dbh->do("use $db");
my $tables = $dbh->selectcol_arrayref("show tables");
for (1..2) {
for my $table (@$tables) {
print "flushing $db.$table\n";
$dbh->do("flush table $table");
}
}
}
print "flushing all\n";
$dbh->do("flush tables");
The Chicago Perl Mongers are excited to officially open the call for participation for YAPC::NA 2008. To submit your proposal, visit the YAPC site, create an account, and let us know what you'd like to talk about. Submissions will be accepted through March 15th 2008, so get yours in soon.
We are currently accepting proposals for conference talks with durations of 20, 45, 70, and 95 minutes. These talks can be directed at any level of Perl programmer, from the noobies to seasoned Perl veterans. If you want to show off an amazing hack or new framework that youve been involved with, this could be your chance. Possibly you just want to contribute to the community by giving an introductory talk on regular expressions or subroutines; we intend on having something for everyone this year.
If 20 minutes is a too intimidating for you right now or if you just dont have enough material to fill up that much time, dont fret, we'll be opening the call for lighting talks soon. These 5-minute mini-presentations are a great way to get a quick point across and to get some experience in presenting in front of a crowd. Keep watching this news source for more information about the lighting talks.
This year we are also planning on introducing more hands-on workshop-style tracks to the conference. These sessions will typically be a little longer than a normal presentation and will be much more informal. During the workshops, conference attendees will be able to interact with presenters to actually do things like compile Parrot or create a hello world program in Perl 6. If you are involved in a project and would like to host a workshop, please contact Josh McAdams directly at joshua dot mcadams at gmail dot com.
An easy recipe to do this is to include a dependency on a module version which only development releases reached that far.
That will provoke installations by the CPAN shell to fail complaining about a bad dependency. These actually will generate NA test reports because of dependencies which could not be satisfied.
I saw such an example the other day. The dist KSx-Analysis-StripAccents
declared a dependency on 0.2 version of KinoSearch
. The latest stable release of KinoSearch is 0.162 and the current devel release is 0.20_05. That filled CPAN Testers with NA reports on SPROUT's module.
Note that this is not a major fault from author's part. It is just how the toolchain works nowadays. There is no automated way to have a dependency on development versions, which seems a good thing, but which cannot be circumvented (unless it is done manually).
Maybe, that has some resemblances with declaring dependencies on third-party modules which are not in CPAN (like SVN::Core
and modules in the list kept by Module::ThirdyParty,
and company-specific code).
When testing/playing with a distribution, one usually runs:
$ perl Makefile.PL; make
$ perl -Mblib demo.pl
# or (with tests)
$ prove -b test.t
Sometime the make
(or ./Build
) gets annoying and a shortcut is nice:
$ perl -Ilib demo.pl
# or
$ prove -l test.t
However, there are a bunch of reasons not do that. Among them:
-Ilib
won't work for modules with XS parts (it will use installed XS components or fail to load)
lib/
layout (with modules not in the MANIFEST and things like that)
At the end, we conclude that working under -Mblib
is safer (and closer to release conditions) than playing with -Ilib
.
Tomorrow at this hour I’ll be asleep 400km from home.
At $work, an application that I wrote (remora, a companion application to Wireshark) is extensively using NetPacket. NetPacket is very handy but, alas, has some bugs, and seem to be orphaned (the latest update is from 2003). I've tried to poke the maintainers, but Stephanie Wehner's email is no longer working, and Tim Potter didn't reply yet (although, to be fair, I only began to poke last week).
Bottom line, I'm playing with the idea of taking over the module so that I can churn a new version with the bug fixes. So if anyone out there is in contact with Stephanie or Tim, can you let them know a madman is nefariously organizing a coup? Thanks. :-)
A while back, around November 2007, there was some discussion about performance problems with rt.cpan.org to the point that some found it to be almost unusable. I bring this up here because there were some suggestions that perhaps TPF could buy some new hardware. If you scroll to the bottom, you'll see Jesse Vincent jumped in and said he thought the performance wasn't a hardware problem and he could make some changes, but he needed help locating the trouble spots.
I'm happy to report that Jesse's team, led by Ruslan Zakirov, has implemented some changes and performance seems to be much better now. Even better, many of the changes made for rt.cpan.org have filtered into the regular RT codebase as of RT 3.6.6. Jesse also told me they are working to publish all the bits of rt.cpan.org that are still
locked away so that the perl community can contribute more actively.
I just wanted to say thanks to Jesse (and Ruslan) for following through on the fixes. I also want to thank Adam and others for pointing out the problems so we could address them. A public service isn't really a service if people can't use it.
Note: I applied some updates after I first published this.
The Association for Computing Machinery has given it\s highly prestigious A.M. Turing Award to the researchers credited for inventing Model Checking: Edmund M. Clarke, E. Allen Emerson, and Joseph Sifakis.
I believe it was in the 1960s that a few computer science pioneers such as Edsger Dijkstra suggested checking computer programs through formal proofs. For instance, it's obvious you'd better be careful if you write code such as:
if ( var > 0 ) func(); else if ( var = 0 ) return;Any programmer looking at that has to ask: what does the program do if var is less than 0? Is it OK if the program just falls through to the next statement? If not, can you be sure that var can never be less than 0 when the program reaches this point? These sort of questions are what formal proofs are supposed to expose and resolve.
Formal proofs looked great for academic exercises, but quickly became untenable when applied to real-life applications because of the explosion of possible states. The same originally was true for Model Checking when it was invented in the 1980s.
Another problem with these systems is that they beg the question of whether a program is correct. You can't run a proof without specifying your requirements in a highly formal manner, and the formal statement could just as easily have a bug as the program you want to check, a qui custodiat custos situation.
Gradually these problems were overcome to enough of an extent that Model Checking proves useful in a wide range of situations nowadays, and a huge number of products support it. It's particularly useful in hardware, probably because you're not facing the irony of using code to check code (the qui custodiat custos situation).
So computer science can solve some of the big problems in quality. Congratulations to the honored researchers.
For those of you using the star seven CSS hack to target current or older versions of WebKit, this parsing bug has been closed in the latest WebKit nightlies. Acid3 specifically tests for this, so any browser that wants to be compliant with Acid3 will have to fix this CSS parsing bug.
For more information about this hack, see:
Dear cpan-testers recipients,
Effective Monday night, Feb 11th, 2008, after over a million test reports[1], we will no longer be supporting the delivery via email of cpan-testers test reports. With an influx of over 3000 messages a day, this was causing us to deliver almost 200,000 outbound messages, most of which nobody read.
cpan-testers test report submission is unaffected by this change.[2]
If you are currently subscribed to cpan-testers, you'll be unsubscribed, and your mail server will breathe a sign of relief.
You can find test reports at:
http://www.nntp.perl.org/group/perl.cpan.testers/
http://www.nntp.perl.org/group/perl.cpan.testers/rss/posts.xml
nntp://nntp.perl.org/perl.cpan.testers/
http://testers.cpan.org/
Why are we doing this? Things were chugging along smoothly, except every few weeks we'd have a problem where a recipient's mail server would stop accepting mail, and we'd very quickly end up with a large backlog of thousands of messages waiting to be delivered. This would slow down delivery for all of our mailing lists. Recently, this has been happening more and more often. We analyzed the
problem, and came to the conclusion that the best solution was to discontinue outbound email. This will let us focus more time on maintaining more important things.
If you have something that is dependent on receiving all of these emails, please let us know.
Thanks!
-R
Footnotes:
[1] http://www.nntp.perl.org/group/perl.cpan.testers/2008/01/msg1000000.html
[2] There are some volunteers working on a new system that will take email out of the picture altogether, creating a faster, more streamlined, and more consistent test database.
Didn’t it used to be the case that when you used the Mac OS X Finder to
burn a CD-ROM that you could then mount that CD-ROM on a Windows box? In the
last few months, I’m suddenly finding that this is no longer the case. So now
I have to use hdiutil
to convert a .dmg file to the
Joliet and ISO9660 file systems:
hdiutil makehybrid -o image.iso -joliet -iso image.dmg
And then I could burn a CD readable on Windows. What the fuck? I burned three CDs that were then useless to me before I finally dug up this hint. And I had this problem with CDs burned by Tiger, too, last summer, so it’s not just Leopard. It seems to me that Mac OS X should always default to building a hybrid CD that’s then readable by Windows, Linux, and everything else. Why doesn’t it?
Something people often want to do is add methods to a specific instance of a class. For example, if you have a web request, you might want to introspect it and add a "deserialize" method if it's a REST request with serialized data. However, you don't want other instances of the Request class to have that method. In this article, we'll see how such a thing is possible.
Let's start with a simple (and contrived) example:
package Foo; use Moose; has 'value' => (is => 'ro', isa => 'Any'); 1;
You can use the class like this:
my $foo = Foo->new( value => 'Hello' ); say $foo->value; # Hello
It sure would be nice to say $foo->say
though. Let's write a
role that implements the say
method:
package Say; use Moose::Role; sub say { say shift->value } 1;
Right now, this role doesn't do much:
$foo->can('say'); # undef $foo->does('Say'); # undef $foo->say; # dies
To make $foo
do Say
, we need to apply the Role. If we were
applying the role to every instance of the Foo class, we would say:
package Foo; with 'Say'; ...
Then Foo->new(value => '...')->say
would work.
We only want to apply Say
to a specific instance,
however. Fortunately, that is possible:
Say->meta->apply($foo); $foo->say; # Hello $foo->can('say'); # CODE(0xc0ffee) $foo->does('Say'); # true my $bar = Foo->new; $bar->can('say'); # no $bar->does('say'): # false
That's all there is to it. Applying Say
to every instance is also
possible at runtime:
Say->meta->apply(Foo->meta);
Let's look at a more complex example:
package Integer; use Moose; has 'value' => (is => 'ro', isa => 'Int'); sub one_less { my $self = shift; return (blessed $self)->new( value => $self->value - 1 ) } sub multiply_by { my ($self, $other) = @_; return (blessed $self)->new( value => $self->value * $other->value ) } sub is_zero { shift->value == 0 } sub say { say $_[0]->value; $_[0] }
Here we have a basic Integer class. You can do things like:
my $two = Integer->new(value => 2)->say; # 2 my $four = $two->multiply_by($two)->say; # 4 my $zero = $two->one_less->one_less->say; # 0 say "is zero" if $zero->is_zero; # is zero
Straightforward. Now let's say we want to add a factorial
method.
We again begin by creating a Factorial role:
package Factorial; use Moose::Role; requires 'is_zero'; requires 'multiply_by'; requires 'one_less'; sub factorial { my $number = shift; return Integer->new( value => 1 ) if $number->is_zero; return $number->multiply_by($number->one_less->factorial); }
Notice that this time we make sure we have all the methods that
factorial
requires by using the requires
keyword. This is
checked when we apply the role.
Let's do that:
my $ten = Integer->new(value => 10); eval { say $ten->factorial } or say "dies"; # dies Factorial->meta->apply($ten); $ten->does('Factorial'); $ten->factorial->say; # 362880 my $ten2 = Integer->new(value => 10); $ten2->factorial->say; # dies
It works just like you'd expect. However, we are doing something
tricky. Let's take a look at the one_less
method. It takes the
current value, subtracts 1, and then wraps up the result in another
Integer
object.
But, you'll note that instead of Integer->new
, we say (blessed $self)->new
. That's because to calculate the factorial,
the result of one_less
also needs to does('Factorial')
. But,
role application only affects one instance, so just saying Integer->new
would fail.
The tricky bit is in the implementation of runtime role application. Basically, when you apply a role at runtime, this happens:
sub apply_role_to($invocant) { { package Some::Dynamic::Package::1; use Moose; extends 'The::Original::Class'; with 'The::Role::You::Just::Applied'; } $invocant = bless $class, 'Some::Dynamic::Package::1'; }
That's not valid Perl, but the idea holds. Whatever you passed to
apply
is reblessed into a dynamically generated package that isa
the original class. When we say:
(blessed $self)->new( value => ... );
instead of:
Integer->new( value => ... );
we're making sure that the Integer we return is in that
dynamically-generated class with the role applied to it. That way if
we call one_less
on a class that does Factorial, the instance we
return can also do Factorial.
The key thing to take away is that it's possible, easy, and clean to
add methods (via roles) to specific instances of classes. However,
you do need to make sure your class isn't working against, like our
Integer class would be if we just returned Integer->new
.
Incidentally, did you know you can modify a method's invocant? Try this sometime:
package Foo; sub go { say 'Foo::go'; $_[0] = 'Bar'; } package Bar; sub go { say 'Bar::go'; $_[0] = 'Foo'; } my $class = 'Foo'; $class->go; # Foo::go $class->go; # Bar::go $class->go; # Foo::go
This is a great way to ensure that your code is completely unmaintainable. But it's also good for implementing runtime role application.
Thanks to some help from Will Willis and Jeremy Fluhmann, of the Perl Foundation conferences committee, yapc.org has a few new items. Specifically, we've included ads from the Perl community ad server provided by Gabor Szabo and a new Workshops page that describes how to organize a workshop and what TPF can do to help.
I finally wrapped my CVS-encrusted mind around Git's hooks. Huzzah! The biggest hurdle, really, was realizing that there is no spoon.
Anyway, as I'm an unsalvageable slob who always
forget to run perltidy before committing changes, I've
written a pre-commit
hook that makes sure that
all Perl files to be committed are all clean, neat and tidy
(and aborts the commit and chides me if they are not):
#!/usr/bin/perl
use Perl6::Slurp;
my $status = slurp '-|', 'git-status';
# only want what is going to be commited
$status =~ s/Changed but not updated.*$//s;
my @dirty =
grep { !file_is_tidy($_) } # not tidy
grep { /\.(pl|pod|pm|t)$/ } # perl file
map { /(?:modified|new file):\s+(\S+)/ } # to be commited
split "\n", $status;
exit 0 unless @dirty; # Alles gut
warn "following files are not tidy, aborting commit\n",
map "\t$_\n" => @dirty;
exit 1;
### utility functions ###############################################
sub file_is_tidy {
my $file = shift;
my $original = slurp $file;
my $tidied = slurp '-|', 'perltidy', $file, '-st';
return $original eq $tidied;
}
A familiar and unpleasant sight in web applications is fragile code traversing the DOM to get to certain elements or collections of elements. Chains of getElementById("something").parent.parent
are all too common, hurting both readability and flexibility. As a result, many javascript libraries have implemented functions to use the powerful CSS selector system to look up DOM nodes.
Continuing the trend of standardizing and speeding up commonly used functionality from these libraries, WebKit now has support for the new W3C Selectors API, which consists of the querySelector
and querySelectorAll
methods. Hopefully libraries will begin to adopt this new functionality to provide performance improvements while remaining compatible with older browsers.
Some examples of how it could be used:
/* * Get all the elements with class "hot" (duplicating getElementsByClassName) * A common use for this is as a toggle; * for example, a search feature might tag results with a class */ document.querySelectorAll(".hot"); /* * Get the currently hovered element */ document.querySelector(":hover"); /* * Get every other element in the <li> with id "large" * This is mostly useful for doing "zebra stripe" alternating rows. * Once CSS3 becomes more widespread, doing this directly via CSS will be more practical */ document.querySelectorAll("#large:nth-child(even)");
What would a new API like this be without benchmarks? The mootools project has conveniently created a wonderful test and benchmark suite for just this sort of functionality. A version of it that has been modified to test querySelectorAll as well as three common javascript libraries is available at http://webkit.org/perf/slickspeed/. Please note that many of the tests will only work in a build of WebKit from February 7th or later.
Perl will be Y2038 safe. And yes, I'm going to get it backported to 5.10.$ ./miniperl -wle 'print scalar localtime(2**35)'
Mon Oct 25 20:46:08 3058
For the last several years, I’ve run a Courier-IMAP mail server for all of the mail for this site, Kineticode, Strongrrl and other domains. We mainly used Mail.app on Mac OS X to communicate with the server, and it worked really well. Today, Julie has over 3 GB of mail data, and I have around 1.5 GB, all managed via IMAP.
Recently, I decided it was time to move the mail elsewhere. I’ve been meaning to do it for a while, primarily because the server I was using is now used for the Bricolage project, and because I never set up any spam filtering. Julie was suddenly getting 100s of spam messages in her inbox. (It really didn’t help that she was still using Panther.) So on the advice of a good friend who had been evaluating various mail services—and who for now shall go nameless and therefor blameless—I moved all of our mail to FuseMail.
At first this seamed like a pretty good solution. Our spam rates went way down, I could set up unlimited mail lists, aliases, and forwards, and there was a migration tool that automated moving all of our existing mail from the old IMAP server to the new one. There were some glitches with the migration tool, but in the end all of our mail was moved and in tact.
But that’s when I started to notice the issues. To summarize:
Sent Itemsfolder by Mail.app was marked as unread. This didn’t happen on the old server, and apparently has something to so with how FuseMail names the sent folder:
Sent Itemsrather than
Sent Messages.
Sent Itemsinstead of the traditional
Sent Messagesfor all sent mail, I asked if they could move the 1.8 GB of messages from Julie’s Sent Messages to their Sent Items, since Mail.app would just choke on such a task. Though my request was escalated to the FuseMail developers, the answer came back
no.Which I guess means that they’re not using Maildir, because in that case it would be a cinch, n’est pas?
Return-Path
header
modified. This made RT break until I hacked it to ignore that
header (which is its by-default preferred header for identifying
senders.So I’m pretty fed up. It took me a week to get all of our mail on FuseMail, and now I’m looking at moving it off again (once OfflineIMAP finishes a full sync). Grr. I’m considering finding a virtual host somewhere and setting up my own IMAP server again, but then I have the spam problem again. So then I could use a forwarding service like Pobox, or I can set up my own spam filtering (something I had hoped never to get into managing myself). My old IMAP server required very little maintenance, which was nice, but then the span filtering stuff always seemed daunting. Don’t you have to update things all the time?a
But before I go off and do something else, and unlike before I moved to FuseMail, I wanted to get an idea what other folks are doing? Do you use IMAP? Do you use it to manage a shitload (read: Gigabytes) of mail? Do you get very little spam and still get all of your valid mail? Are IMAP folder maintenance actions fast for you (in Mail.app in particular)? Are you paying a not-unreasonable amount of money for your setup? If you answered yes to all of these questions, please, for the love of all that is good in this world, tell me how you do it. I’m looking for something that I don’t have to work very hard to maintain (hence my original attempt to have some company that specializes in this stuff do it), but I’ll do what I have to to make this thing right. So how do you make it right? And if I have to run my own server, where should I host it that won’t cost me an arm and a leg?
Thanks for your help!
I quite enjoyed reading Brent Ashley’s description of his travails when trying to set up a Windows server
When I finally get through a marathon session of setting up a Windows server “just so” to make my application work, it’s like I have created a finely balanced stack of Jenga blocks from which I back away slowly on tiptoe, praying it doesn’t tumble. I’ve get very little confidence that fixing anything will be an excercise in logic and deduction more than having to know some obscure incantation or uninstalling components wholesale and reinstalling and reconfiguring them.
Escalation wars... you just have to love'em.
First, David came up with the über-cool CPAN deps page.
Then Andy comes up with a nifty Greasemonkey script to add it to the CPAN distributions main pages.
Then I add a small patch to the script to retrieve some information from the Deps page.
Then David creates an xml interface to CPAN deps, opening the door wide open for Web 2.0 goodiness.
Then (and this is where we are right now) I hack together a new CPAN_Dependencies monkeyscript to take advantage of said xml interface.
This, of course, is nowhere near the end of the story. My new script only scratches the surface of what can be done with the information contained in the xml. As soon as I have some tuits, I'll probably add a way to toggle between showing only the first-level dependencies and all dependencies, and have dependencies color-coded by degree of b0rkage, and whatever bell or whistle I can think of in the meantime.
I’d like to thank Shlomi Fish for taking the time to transcribe the Perlcast interview with Tom Limoncelli about “Time Management For System Administrators”
So I did it. I proposed to the perl5-porters that we should enable "use strict" by default for some future code. This may be a little less preposterous than it sounds at first, so please wait with hitting the "reply" button until you read the whole of this.
My proposal basically goes as follows:
I'll include my original rationale here:
Personally, I've always wanted perl to have strictures on by default for my code. I would think that 95% of all code bases which were written in this century and which are of non-negligible size import "strict". I don't use strictures for one-liners, of course, but for anything else it's a must. It seems to me like others have similar views on this. Try posting some code without "use strict" to some newsgroup or forum and ask for help. Make sure not to give out your email address, though.
"use 5.10.0;" already auto-imports feature.pm and loads the 5.10 specific features.
How about having "use 5.11.0;" (or 5.12.0) automatically import strict along with the 5.10+5.11 feature set? Naturally, the -E switch for one-liners should *not* do that.
This would *not* break backwards compatibility. This would not affect one-liners. This would optimize for the common case: If you write enough code to make importing optional features worthwhile, odds are very high you'd be importing "strict" anyway. The 5% who need to disable strictures again can still add a "no strict;" statement.
strictures-correct code has been best-practice for a long time now. Let's make it the default for *new* code.
Just think of strictures as a feature. It just makes sense.
To my surprise, the proposal has been received with very positive feedback. So I wrote the patch.
With some luck, we'll get strictures by default in 5.12! Flame away!
Cheers,
Steffen
In my journal entry from December 22, 2006, I said I'd used a few hacks to package parinstallppdgui into an executable binary using PAR::Packer. I'll explore that further here:
parinstallppdgui is mostly a GUI front-end to parinstallppd. It doesn't use PAR::Dist::InstallPPD internally, but uses the system to run parinstallppd so if the child process bombs out, the GUI process can show a friendly warning to the user (comprised of the STDERR and STDOUT of the child process) instead of crashing. That's all very simple if you have perl on your system, of course.
Now, if you want to package parinstallppdgui or any other Perl script that uses another Perl script into an .exe, you'll quickly find out that without a perl on the target system, these two scripts cannot share the same interpreter. The obvious "solution" is to package both of them up into an .exe separately and ship them both. This has several problems. First, you need to ship two executables instead of one. Second, the first executable won't necessarily know where to look for the second if the user doesn't put them in PATH. Adding a couple of FindBin hacks isn't great at solving this, either! Third, these two executables will have a lot in common - starting with all the core Perl modules.
So I took a slightly better, yet more complicated route. The process is as follows:
if (defined $ENV{PAR_TEMP}) {
require Config;
require File::Spec;
$ENV{PATH} .= (defined $ENV{PATH} ? $Config::Config{path_sep} : '')
. File::Spec->catdir($ENV{PAR_TEMP}, 'inc');
$ENV{PAR_GLOBAL_TEMP} = $ENV{PAR_TEMP};
}
pp -o parinstallppdgui.exe parinstallppdgui
pp -o parinstallppd.exe -l expat parinstallppd
pare -u parinstallppdgui.exe parinstallppd.exe
pp -o parinstallppdgui.exe -a parinstallppd.exe -l expat parinstallppdgui
Cheers,
Steffen
P.S.: A third and even better solution might be to package both of the scripts into the same binary and symlink or copy that binary to parinstallppd.exe and parinstallppdgui.exe and let PAR figure out what to run. This is cool if you have symlinks and sucks if you don't.
So CPANTS was down for a couple of weeks. This morning, I noticed it's back, but so far without the cron job that updated the data nightly.
For me, CPANTS has been a way to check my eighty or so CPAN distributions for packaging shortcomings. Unfortunately, that check happens only after I made the release. I installed Module::CPANTS::Analyse which comes with cpants_lint.pl. Given a distribution file name, it runs the CPANTS metrics against it.
Since Module::CPANTS::Analyse comes with quite a few prerequisites, I set up a simple web service which you can use to check your distributions before uploading. The interface is a little cumbersome, but I think it's worthwhile. Just upload your distribution at http://steffen-mueller.net/cgi-bin/cpants-limbo/check.pl and then visit the result URL given by that script after a few minutes.
The reason for the delay is that I didn't want to install all prerequisites on the hosting account and certainly didn't like running cpants_lint.pl from a CGI process. Thus, my local server fetches distributions-to-scan from there and uploads a result text every three minutes.
Cheers,
Steffen
Mindboggling.
The answer to your question is simple. It is the way these politics are played. See Medicare and -aide as examples. It doesn't matter if the first incarnation was so screwed up that its taken forty some years to try and right it, it was the first to make it through the legal system largely intact. Because it has already survived legal challenges and came out the other side with legislative and judicial approval. Any other state that wants to do the same doesn't have to work to get it going at home. All they have to do is copy and paste.
Simply amazing that the effects of the law can be so far away from what was intended (either through accident or design, it matters not which at this juncture) that I am left wondering if anyone, anywhere will pay politically for the screw up.
Well, I’m here in San Francisco and starting to settle in nicely. Lots of social stuff going on, fun work that doesn’t feel like work, and I found a place to live really quickly and easily.
This photo’s by the lovely yarnivore who took it last weekend at a Rock Band extravaganza where we (I say “we”, though I didn’t do much of it) played right through the full 58-song set list. It took about eight hours.
But the real reason I’m posting today is to assist anyone stalking me. I assume you’re already following my photos on Flickr but you might also like to check out my work blog where I’m, well, posting about work things: Freebase, open data, mashups, that sort of stuff. And if you’re going to be around these parts on Feb 6th, be sure to swing by the Freebase user group meeting for pizza, beer, and of course lots of technical presentations.
No TagsFor a long time, the only easy way to submit CPAN Testers reports was CPANPLUS. The first automated smoke testers for CPAN were built upon it, like CPAN::YACSmoke and POE::Component::CPAN::YACSmoke.
About a year and a half ago, I wrote CPAN::Reporter, to bring CPAN Testers reporting to good-old CPAN.pm. Since then, some intrepid individuals have built automated smoke testers upon CPAN::Reporter, all writing their own programs to do so.
CPAN.pm itself added an experimental smoke command to test recent uploads, but it requires XML::LibXML and doesn't test distributions in isolation -- all tested distributions and their dependencies keep getting added to PERL5LIB until things explode.
Yesterday, I released the first, alpha version of CPAN::Reporter::Smoker to provide a better-behaved, more turn-key approach to smoke testing with CPAN::Reporter.
(... configure CPAN and CPAN::Reporter as usual ... )
$ perl -MCPAN::Reporter::Smoker -e start
This initial version still has some limitations (e.g. no developer versions smoked), but I'm ready for anyone interested to try it out and start sending feedback -- compliments, suggestions, or complaints all welcome.
-- dagolden
The minute you start adding custom attributes, things end up well past 70-80 columns wide:sub update : Chained('instance') PathPart Args(0)
sub update : Chained('instance') PathPart Args(0) Form('someform.yml') Template('this/page') ActionClass('REST')
so $VIMRUNTIME/syntax/perl.vim
syn keyword perlStatementStorage state
syn keyword perlStatementFiledesc say
if exists("perl_fold") && exists("perl_fold_blocks")
syn match perlConditional "\<given\>"
syn match perlConditional "\<when\>"
syn match perlConditional "\<default\>"
syn match perlRepeat "\<break\>"
else
syn keyword perlConditional given when default
syn keyword perlRepeat break
endif
if exists("perl_fold")
syn match perlControl "\<BEGIN\|CHECK\|INIT\|END\|UNITCHECK\>" contained
else
syn keyword perlControl BEGIN END CHECK INIT UNITCHECK
endif
Back at the Nordic Perl Workshop 2007, I sat down with Ovid to talk about logic programming. Here is the audio for that interview.
Helix is eligible for nomination in the semiprozine category. A nomination would brighten my summer.
I wouldn’t mind seeing William Sanders up for editor, short form, either.
I would just like to point out to any readers here who happen to be Hugo voters that Helix is eligible for nomination as a semiprozine. If you happen to have your Hugo nominating ballot lying around, and you were thinking, “Gee, what could I nominate, so as not to waste this fine ballot?” — well, there’s one possibility
.
At last!
I believe that every book I owed any donor is now on its way, though how I’m defining that gets a bit tricky.
There are seven sitting in a box in the dining room waiting to be taken to the post office. There’s one on my desk waiting for an e-mail confirming how the donor wanted it personalized.
All the others have been mailed.
Most packages included The Vondish Ambassador, a chapbook of “The God in Red,” and an insert map of the Empire of Vond. I signed or personalized The Vondish Ambassador as requested.
However, I know that in at least one case I sent a signed book to someone who hadn’t wanted it signed, and sent an unsigned copy to someone who had requested a signature. I’m fairly sure that I forgot at least one map. It’s possible I also messed up in other ways. My apologies, but packing and mailing hundreds of books is one heck of a job. I got most of them right.
There were also a few people who I could never get a response from, so those books may have gone to incorrect addresses. I tried, honest.
If you did not receive the map (it should be just inside the front cover), let me know, and we’ll see what we can do about it.
Two people who are entitled to a copy of the book declined — one prefers an e-book and the other is waiting for a mass-market edition.
If your copy of the book hasn’t arrived yet at all — especially if you’re outside the U.S. — give it a few days, but if it hasn’t shown up by Candlemas, drop me a line.
As of right now, there’s still nothing definite about a mass-market edition.
Several people have asked me when I’ll be starting the next serial, and what it’s going to be, so let me once again say, “I don’t know.” It depends on when I get other projects done, what sort of offer Tor makes (if any) for the Fall of the Sorcerers, and various other factors, some of them beyond my control.
My best guess is that I’ll start a third serial late this year — maybe as a Christmas present for my readers. Whether it’s Ethshar or science fiction — well, we’ll see.
Thanks to everyone for your continued support!
I'm going to be presenting the Perl Teach-In in Birmingham on Monday 31st March as part of UKUUG's Spring 2008 Conference. The talk will be (lightly) updated from the one I gave at the BBC last summer.
This tutorial won't be free. As I understand it, the class will only be open to people who are registered at the conference, but I don't currently know how much that will cost or whether you'll be able to buy a ticket for just the tutorial day. I'm sure more information will appear on their web site soon.
I should start looking in to running the Teach-In in London again.
So, when I request a regular old web page, really wanting HTML, the REST controller negotiation falls back to text/xml, even though my default type in config is text/html. Grrr.text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8, image/png,*/*;q=0.5
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Labor Senator Stephen Conroy’s new Internet censorship policy:
Federal Labor will improve existing government programs in this area by:
Providing a mandatory ‘clean feed’ internet service for all homes, schools and public computers that are used by Australian children, so that ISPs will filter out content identified as prohibited by the Australian Communications and Media Authority (ACMA). The ACMA ‘blacklist’ will also be made more comprehensive to ensure that children are protected from harmful and inappropriate online material.
(emphasis mine)
The difference between this load of steaming horse shit and the rather similar load that got dumped on us in 1999 is that this time the mainstream media actually know what’s going on:
When a Queensland newspaper’s headline reads “Nanny Rudd censors the internet”, you know the times are a-changin’. So, I have some hope that this idiotic proposal will either get ditched entirely, watered down to pointlessness (as 1999’s was), or worked around to a significant degree. I won’t be here for it, but good luck with that.
No TagsHappy New Year and welcome back to Perlcast. Though 2007 ended with a very light lineup of shows, we here at Perlcast are excited about what 2008 has to offer. There are plans for more interviews, more presentations, and possibly even some videos and video tutorials about Perl.
Anyway, let’s get back to the podcast. Way back at OSCON 2006, I sat down with Stas Bekman and Philippe M. Chiasson to talk about mod_perl. Though quite a bit of time has passed since the show was recorded, there is still a lot of good mod_perl related information in the show. I hope that you enjoy it.
I seem to be constitutionally incapble of avoiding that subject line. Sigh. I apologise.
In any case, it’s official. I’m moving to California in less than two weeks’ time.
I put off mentioning it here in some sort of fit of superstition. I didn’t want to jinx it. But back in November I received a job offer from Metaweb, Inc to become the Community Director for Freebase. I accepted, and the visa paperwork began. Yesterday my visa was approved, and now the tickets are booked. I fly out on January 15th.
The job is based in San Francisco in the SOMA district, and I’ll initially be staying with friends in the Mission til I find a share house to move into. Plus I’ll get to travel to conferences, so I’m hoping to get to see a bit of the US and the world.
Right now I’m in the throes of packing everything I own into boxes and saying goodbye to people in Australia. Life is crazy but good.
No TagsHelix No. 7 is now up and open for business — check it out.
There are seven new stories, by Charlie Anders, Maya Bohnhoff, Adam-Troy Castro, David W. Goldman, Selina Rosen, Vaughan Stanger, and Laura J. Underwood, along with poetry by Mike Allen, F. J. Bergmann, Anthony Bernstein, Gene van Troyer, and Jane Yolen, and columns by the regulars — John Barnes, Bud Webster, etc.
The only pay the authors of those seven stories will receive is a share of what readers donate, so if you like what you read, do please show that appreciation in tangible form, either through PayPal or by sending a check.
A common pitfall when designing models is the tendency to tie application logic to Catalyst. The problem with this approach is that model classes become difficult to reuse outside of Catalyst, which is a common requirement for most applications. A better design approach would be to write and test a plain class outside of your main web application, and then painlessly glue it to Catalyst with a couple lines of code.
Catalyst gives you a chance to instantiate all your components (Models, Views and Controllers) during load time by calling the COMPONENT() method on each component class. This means you can implement it and return whatever you want from it. For models this boils down to:
package MyApp::Model::SomeClass; use warnings; use strict; use base qw/Catalyst::Model/; use SomeClass; sub COMPONENT { my($self, $c, @config) = @_; return SomeClass->new(@config); } 1;
This particular implementation passes the configuration for this model to the
external class on the call to new(). Of course, you could choose to instance
the class any way you want to. Later on, $c->model('SomeClass')
will get
you the SomeClass
instance, not MyApp::Model::SomeClass
. Notice that
the returned object is the very same instance that was created initially when
your app was loaded.
Every time $c->model
is called, Catalyst gives you a chance to run custom
logic by attempting to run ACCEPT_CONTEXT on the model, whatever is returned
by this method is what $c->model
returns as well. So, if you need a
new instance on every call, your model becomes something like:
package MyApp::Model::SomeClass; use warnings; use strict; use base qw/Catalyst::Model/; use SomeClass; sub ACCEPT_CONTEXT { my($self, $c, @args) = @_; return SomeClass->new(@args); } 1;
So when you call $c->model('SomeClass')
, you'll get a fresh instance of
SomeClass
not MyApp::Model::SomeClass
.
Catalyst::Model::Adaptor
wayInstead of implementing your own glue code, you can use the generic
implementation provided by the Catalyst::Model::Adaptor
module, which
provides several different ways of building your external class instances.
If you need a single application-wide instance of your external class, you can
inherit from Catalyst::Model::Adaptor
:
package MyApp::Model::Foo; use warnings; use strict; use base qw/Catalyst::Model::Adaptor/; __PACKAGE__->config( class => 'SomeClass', args => { foo => 'bar' } ); 1;
Of course, you can also configure your class via myapp.yaml
Model::Foo: class: SomeClass args: foo: bar
This gives you more flexibility when you decide to change your implementation,
just replace SomeClass
with whatever class you wish to use, without even
touching your code.
For instancing objects on every call to $c->model
, just inherit from
Catalyst::Model::Factory
instead. And for instancing objects on a
per-request basis, inherit from Catalyst::Model::Factory::PerRequest
.
The Catalyst::Model::Adaptor
documentation provides information on how to
further customize your models to address your specific needs.
You can easily create glue models by using the helpers provided with
Catalyst::Model::Adaptor
:
script/myapp_create.pl model SomeClass <type> MyApp::Model::SomeClass
Where <type> can be Adaptor
, Factory
or Factory::PerRequest
.
Jonathan Rockway, for writing Catalyst::Model::Adaptor
.
edenc - Eden Cardim - edencardim@gmail.com
This article discusses about two catalyst concepts .
1. How chaining can be used to create applications that have a central engine that process core logic and allow independent modules be built on top.(The Application)
2. Par and creating spawns of the application that can span again and integrate to the master site . The individual spawn can be carried around as a separate application.(Instantiating)
The tutorial is accompanied by a functional and well commented code base which can be checkout using svn from http://dev.catalyst.perl.org/repos/Catalyst/trunk/examples/Quiz
For demonstrating the above mentioned concepts a quiz application has been taken as example.
The final application must be able to
I). Create quizzes that can be taken on the site or deployed to a trainer's laptop , which the trainer can carry to colleges.The application must make sure that the deployed quiz contains only the required questions in the database as the questions can be proprietary and the trainer must be able to take only what is necessary. It must also allow a deployed quiz to act as master and create smaller quizzes from it . Ideal when a trainer takes 500 questions and has to train 10 batches with different question set in the same college.
II). When creating a quiz it must be possible to choose the modules and the number of questions from each module .Example of questioning modules can be "choose the correct answer" ,"Fill in the blanks " etc.
III). It must be possible to build new modules that can work along with the application and yet can be developed independently without any constraints on its internal logic , look and feel. This tutorial will take you step by step on building such an application .
This tutorial will take you step by step on building such an application .
What is needed is a quiz engine that can create quiz containing questions from more than one module . And be able to track who took the quiz and what they scored.
The sql and the schema to create the required tables and models can be seen from the code. Basically there are 4 tables.
QuizMaster::Quizzes stores the generated quizzes. QuizMaster::Modules stores the list of available modules. QuizMaster::QuizModules stores the modules selected for a quiz , and the randomly assigned question numbers from each module. QuizMaster::participants stores who took what quiz and how much they scored.
Controller methods (index and create ) are written in quiz controller the general way to display the created quiz and to allow to create a new quiz.
What is expected is that the application act as a quiz engine that controls modules to ask questions .The modules are designed to ask questions and get answers in whatever manner is best suited for that module and pass back only the result to the engine so that the engine can track the score and the flow.
For example : A quiz might consist of two module "Choose the correct answer " , "Fill in the blanks".
Both the modules may have completely different UI , a module may select more than one value as an answer. A quiz can contain questions (1,2,8) from module 1 and (4,6) from module 2.
The application has to be designed in such a way that the flow , i.e the transition from question 1 to question 2 when in module 1 and the transition from question 8 in module 1 to question 4 in module 2 is handled by the engine.The engine must also handle the score manipulation.
Even when doing the above , the application must still keep it easy to create new modules.
This is possible by combining sessions and chaining in the following way .
We define 3 methods in the quiz controller to do the above.
(start, process and finish)
Each of which are explained below .
This function does two things 1.Initializes the quiz by loading the details of the modules used and their respective question numbers into the session. 2.Determines the first module and the first question that have to be asked and its corresponding url.
sub start : Local {
my ($self,$c,$quizid) = @_; #Displays a form and gets the participants name. if ($c->request->param('name')) { #Loads the modules and the questions selected randomly from each module #during create and stores it in session my $quiz = $c->model('QuizMaster::Quizzes')->search({id=>$quizid})->first; my $modules = [$quiz->search_related('quizmodules',{quiz=>$quizid})->all]; $c->session->{quiz}={participant=>$c->request->param('name'),name=>$quiz->name}; my $moduledata; foreach my $module (@{$modules}) { push @{$moduledata} , { id => $module->id , name=>$module->module->name, questions => [split ' ',$module->questions] }; } $c->session->{quiz}->{modquestions}=$moduledata; #set up counter for the questions and modules $c->session->{quiz}->{modindex}=0; $c->session->{quiz}->{qindex}=0; #set score and total score as 0 $c->session->{quiz}->{score} = 0; $c->session->{quiz}->{tscore} = 0; #Sets up the url for the first question of the first module. $c->stash->{newurl} = "/quiz/process/$quizid/$moduledata->[0]->{name}/$moduledata->[0]->{questions}->[0]"; $c->stash->{template} = 'quiz/begin.tt2'; }
}
This method plays the key role of maintaining the flow and scoring
This controller does two things . 1.It does scoring for the previously answered question 2.Set the action path(url) for the next question that can be used by the module displaying the current question
sub process : PathPart('quiz/process') Chained('/') CaptureArgs(1) {
my ($self,$c,$quizid) = @_; #Scoring $c->log->debug($c->request->method); if ($c->request->method =~ /POST/) { $c->log->debug('Entered here'); # To make sure we don't evaluate a non existing previous answer when displaying the first question if ($c->session->{quiz}->{answer}) { my $flag= 1; # The questioning module has to store the answer for the question in the application's session # Using a hash here allows to have questions with multiple answers foreach (keys %{$c->session->{quiz}->{answer}}) { if ($c->session->{quiz}->{answer}->{$_} != $c->request->param($_) ) { $flag = 0; } } # The score per question is taken as 10 here . #It can also be set to take a value from the sessions in which case the #questioning modules can assign different marks to each question my $spq = 10; $c->session->{quiz}->{score} += ($flag * $spq); $c->session->{quiz}->{tscore} += $spq; $c->log->debug($c->session->{quiz}->{score}); } } #Determine the action path(url for next question) my $modindex = $c->session->{quiz}->{modindex}; my $qindex = $c->session->{quiz}->{qindex}; #check that the current question is not the last question of the module if ( $qindex != $#{$c->session->{quiz}->{modquestions}->[$modindex]->{questions}}) { $c->session->{quiz}->{qindex} += 1; $qindex ++; # generate /quiz/process/quizid/modulename/questionid my $modulename = $c->session->{quiz}->{modquestions}->[$modindex]->{name}; my $questionid = int($c->session->{quiz}->{modquestions}->[$modindex]->{questions}->[$qindex]); $c->stash->{postaction}="/quiz/process/$quizid/$modulename/$questionid"; } else { #if it is the last question , check that the current module is not the last module if ($modindex != $#{$c->session->{quiz}->{modquestions}}) { $qindex = $c->session->{quiz}->{qindex} = 0; $c->session->{quiz}->{modindex} += 1; $modindex ++; my $modulename = $c->session->{quiz}->{modquestions}->[$modindex]->{name}; my $questionid = int($c->session->{quiz}->{modquestions}->[$modindex]->{questions}->[$qindex]); $c->stash->{postaction}="/quiz/process/$quizid/$modulename/$questionid"; } #if the current question is the last question of the last module set action path to the quiz finish controller else { $c->stash->{postaction}="/quiz/finish/$quizid"; } }
}
This method does two things
1. Manipulate the score for the last answer. 2. Save the quiz results to the database.
(Actually this can be re-factored to chain that process method and hence pop point 1 from its responsibility .It will be necessary to do so when complex scoring algorithms are used in the quiz . Say for example each module has a different scoring patter for questions and a that module allows multiple answers and different scores for each answer.In such a case the scoring algorithm will be complex and it will not be a good design to repeat that portion in the controller below for the last question alone)
sub finish : Local {
my ($self,$c,$quizid) = @_; my $name = $c->session->{quiz}->{participant}; # Same as in the previous controller , manipulate the score for the last question. my $flag= 1; foreach (keys %{$c->session->{quiz}->{answer}}) { if ($c->session->{quiz}->{answer}->{$_} != $c->request->param($_) ) { $flag = 0; } } my $spq = 10; $c->session->{quiz}->{score} += ($flag * $spq); $c->session->{quiz}->{tscore} += $spq; my $score = $c->session->{quiz}->{score}; my $tscore = $c->session->{quiz}->{tscore};
# Save the Quiz results
my $participant = $c->model('QuizMaster::Participants')->create({ name => $name, quiz => $quizid, score => $score, totalscore => $tscore }); $c->stash->{score} = "$score / $tscore";
}
Having designed the engine above. There are few things to be kept in mind to create a questioning module .
1. The module model must have the questions in the namespace modelname::modelname_questions (This can be avoided if the Quiz::Modules table can have the count of the total number of questions available for each module or maybe a field that contains the name of the model that the module uses for storing its questions ).
1.There must be a controller with the same name as the module 2.The questionid of the question that is to be challenged will be given to the module's method that is chained with the quiz controller's process method and has PathPart same as the module's name. 3.The form action value(url for the next question) will have to be retrieved from the session. 4.The answer(s) to the current question must be stored in the session as a key/value pair.
When the above points are followed ,
The three methods in the quiz controller (start , process and finish) handle scoring , moving over the questions and moving over to the next module .
With that in mind, the complete controller code for a simple choose the correct answer module is
sub index : PathPart('Ctca') Chained('/quiz/process') Args(1) { my ( $self, $c , $questionid ) = @_;
my $question = $c->model('Ctca::CtcaQuestions')->search({id=>$questionid})->first; $c->stash->{question} = $question; $c->stash->{options}= [$question->search_related('options',question=>$questionid)->all]; # The only extra line by the module that is needed to let the engine handle the flow and scoring $c->session->{quiz}->{answer} = {choice => $question->correct}; $c->stash->{template} = 'Ctca/index.tt2';
}
The above architecture reduces the work involved in designing a new module . When creating a new module one will have to only worry about
* How the question is displayed and answer taken (drag and drop , jigsaw , draw lines :)) * Determine the correct answer from the database and store it in the session
Everything else is taken care by the quiz controller .
This makes it easier to develop new modules .
This part of the tutorial outlines the step required in creating spawns of the application that can span again and integrate to the master site . The individual spawn can be carried around as a separate application.
The application is so build from above that
1.If we were to copy the entire folder to a new folder . 2.Delete all the records in the QuizMasterL::Quizzes table except the one selected for deployment. 3.Delete all the records in each quiz module other than the ones used by the quiz 4.Create par and use pp to make a binary with the perl compiler.
We will have a complete spawn of the existing site with a limited number of questions. Hence a trainer can create sub quizzes from the limited number of quizzes and that runs easily with a single click.
We will use a method called deploy in the quiz controller which does the above 4 points. Since it is easier to handle steps (1,4) in a script run locally . The controller script will first do steps (2,3) and save the created databases to a temporary location. Then the system command can be used to invoke a shell script that will do the copy and packaging.
That is to create a slice of the existing database the following procedure is followed.
1.Use deploy() to create the database for the QuizMaster::Quizzes . 2.Insert a record with the quiz id and the name to that database. 3.use deploy() to create the database for each of the module. 4.Read the assigned questions for each module from the QuizMaster::QuizMoules tables and insert those record into the newly created database.
After doing which the control can be passed to the shell script that will do the copy and packaging.
Since we had stated that using the above said convention for the questions table is the only constraint for the module's Model.It is only possible to slice the questions table .It is not possible to know the logic used by the questioning module to store the answers.So we cant slice them. Anyways that is alright as though the questions can make sense without the answers and need to be protected when proprietary , the answers without the questions are just junk and will not make much meaning.
Special thanks to Ma Foi Academy for allowing me to expose part of the code developed for the organization in this article.
Antano Solar John
DBIx::Class::Tutorial::Part1
This is part one of a DBIx::Class tutorial I started writing for the folks at work, I hope to turn it into a multi-part releasable document.
See DBIx::Class::Manual::Glossary.
Some terms needed for this doc:
A DBIx::Class::Schema object is created by calling ->connect
on our Schema subclass, and passing it the DSN (and other possible
values, see DBIx::Class::Schema). Eg: my $schema =
Breadcrumbs::Schema->connect('dbi:mysql;dbname=Breadcrumbs',
'user', 'passwd');
Creating this object does not attempt to connect
to the database yet.
NB: To make use of an existing mechanism to store credentials for
databases, we can do: my $schema = Breadcrumbs::Schema->connect(
sub { My::Util('Breadcrumbs') } );
.
A DBIx::Class::ResultSet, an object which represents a database
query, including all conditions/clauses. Creating one does not run the
associated query, it is stored and used when actual data objects are
requested. The simplest form of ResultSet represents an entire table,
and can be retrieved from the schema object like this: my $rs =
$schema->resultset('Path')
.
A class that describes a table, it's columns and it's relations with other tables.
A DBIx::Class::Row representing an actual row of data resulting from a database query. This could be and entire row, or just certain fields as restricted by a ResultSet.
To create a new set of DBIx::Class DBIx::Class::ResultSource classes, you can either write them by hand, or create from an existing database.
DBIx::Class::Schema::Loader's make_schema_at
can extract schema
definitions from existing databases and create a complete
DBIx::Class schema for you, example:
perl -MDBIx::Class::Schema::Loader=make_schema_at,dump_to_dir:./lib -e 'make_schema_at("Breadcrumbs::Schema", { debug => 1 }, [ "dbi:mysql:dbname=Breadcrumbs","user", "passwd" ])'
This will create a file for each database table (in lib/Breadcrumbs/Schema/), plus the file Breadcrumbs/Schema.pm which is the DBIx::Class::Schema file.
The table files will contain information about the columns and their types. If the database is innodb, it will also extract relationships based on foreign key references etc. An md5 sum of the contents is added, so that the user can add more relations or other methods to the file, and a later update will not overwrite them.
Re-run the make_schema_at
command shown above.
If your databases is mysql and not using innodb, you will have to add the table relationships ourselves. These are the most useful part of DBIx::Class, otherwise we might as well just use DBI..
There are 3 main/useful relationship types, for the rest and a more wordy description, see DBIx::Class::Relationship.
The name
of each relationship (the first argument), is important,
it is used both as an accessor and as a key to joni across tables.
Breadcrumbs's Name
table has a PathID
column which contains
an ID from the Path
table. To make the Path object simple to
retrieve and update, we add the following to the Name.pm file,
after the md5 sum from Loader:
__PACKAGE__->belongs_to('path', 'Breadcrumbs::Schema::Path', 'PathID');
Read as: The value in the PathID
column belongs_to the table
represented by Breadcrumbs::Schema::Path
.
This creates an accessor path
, which we can call on a Name
DBIx::Class::Row which will retrieve the appropriate
DBIx::Class::Row in the Path
table.
## Print the path of the current name entry print $name->path->path;
We can also set a PathID for a new Name row by calling:
$name->path($pathobj); $name->update;
Going in the opposite direction, each Path
row has 0 to many
Name
entries associated with it, indicated by the PathID
column
in the Name
table. We can make it simple to retrieve all the name
values for a particular Path
row:
__PACKAGE__->has_many('names', 'Breadcrumbs::Schema::Name', 'PathID');
Read as: This class has many names
in Breadcrumbs::Schema::Name
linked via the PathID
column.
Creates us an accessor names
which can give us a
DBIx::Class::ResultSet (of which more later), or an array of
Name
objects.
## find all names for current path entry foreach my $name ($path->names) { print $name->name; }
Add a new name for the current path, assuming we have a language object as well:
$path->create_related('names', { name => 'Products', lang => $lang } );
Whether fetching complete rows, searching for rows, or fetching data from multiple tables, the methods are much the same, all return DBIx::Class::Row objects eventually.
To fetch a full row using it's Primary Key (they all have PKs, right?), we can retrieve a Row object directly from a ResultSet representing the table:
## Just PK: my $name1 = $schema->resultset('Name')->find(1); ## More meaningful: my $name1 = $schema->resultset('Name')->find({ id => 1 });
The Row object contains all the values of all the fields defined in the ResultSource. Not necessarily in the table. Each column has an accessor which is by default the name of the column lower-cased (When using Loader).
my $namename = $name1->name; ## Japan my $langid = $name1->langid; ## 1
To fetch a particular Name row and it's Path at the same time, using only one database query:
my $name_rs = $schema->resultset('Name')->search( { 'me.id' => 1 }, ## WHERE { prefetch => [ 'path' ] } ## JOIN AND SELECT. NB: Rel name!
This creates a DBIx::Class::ResultSet, to retrieve the actual Row object:
my $namerow = $name_rs->next;
The following SQL is executed:
SELECT me.ID, me.Name, me.LangID, me.PathID, path.ID, path.Path, path.Parent, path.Position, path.MenuID FROM Name me JOIN Path path ON ( path.ID = me.PathID ) WHERE ( me.id = ? ): '1'
As with the find, the data is retrieved using the column-name accessors. To retrieve a Row object representing the path:
my $pathrow = $namerow->path; my $actualpath = $pathrow->path; ## companyinfo/careers/jp my $pathparent = $pathrow->parent; ## 36
To fetch just a few columns:
my $name_rs = $schema->resultset('Name')->search( { 'me.id' => 1 }, ## WHERE { select => [ qw/id name/ ], ## SELECT as => [ qw/id name/ ], }
This will only select the ID and Name columns. The as
attribute names the columns.
To add more complex conditions:
## All names where the patch matches '/support%' my $name_search = $schema->resultset('Name')->search( { 'path.path' => { 'like' => '/support/%' }, ## WHERE path.path LIKE '/support/%' }, { 'join' => [ 'path' ], ## JOIN }
To insert new rows, call create
on a ResultSet object. This will
first instantiate a new Row object, then call insert
on it. This
can be done individually if needed.
## INSERT INTO .. my $newname = $schema->resultset('Name')->create( { name => 'Support', pathid => $pathid, langid => $langid, }); print $newname->in_storage(); ## 1 (TRUE) my $newpath = $schema->resultset('Path')->new( { path => 'support', }); print $newpath->in_storage(); ## 0 (FALSE) $newpath->insert(); print $newpath->in_storage(); ## 1 (TRUE)
The column-name accessors used to fetch data from a row can also be used to set new values, eg:
## Nonsensically change the language of a given Name row my $namerow = $schema->resultset('Name')->find({ name => 'Japan' }); $namerow->langid(2); $namerow->update; ## UPDATE
Create a resultset containing the condition to select all the rows to
update. Calling update
on the resultset will run the UPDATE
command using the condition in the resultset.
## All name rows with value 'Evaluation' my $name_rs = $schema->resultset('Name')->search( { 'me.name' => 'Evaluation', }); $name_rs->update({ name => 'Free Trial' }); ## UPDATE .. WHERE
Just call delete
on a Row
object.
## No more products/es ! my $pathrow = $schema->resultset('Path')->find({ path => 'products/es' }); $pathrow->delete;
This will also delete related rows that would otherwise be inconsistent. It will remove them *after* the main row. This cascading can be turned off on a relationship if necessary.
Call delete
on a ResultSet object instead. This will run a
DELETE
statement with a WHERE
clause, and will not cascade the
deletes.
my $path_rs = $schema->resultset('Path')->search( { 'me.path' => 'products/es', }); $path_rs->delete; ## DELETE .. WHERE
To delete multiple rows with cascading, call delete_all
on the
ResultSet, which will delete each row and it's related rows
individually.
$path_rs->delete_all; ## many DELETE statements
Add some sorta table descriptions to this doc.
Patches and suggestions welcome.
Jess Robinson <castaway@desert-island.me.uk>
This is a copy of the official announcement about Perl 5.10.
Today the Perl Foundation announces the release of Perl 5.10, the first major upgrade to the wildly popular dynamic programming language in over five years. This latest version builds on the successful 5.8.x series by adding powerful new language features and improving the Perl interpreter itself. The Perl development team, called the Perl Porters, has taken features and inspiration from the ambitious Perl 6 project, as well as from chiefly academic languages and blended them with Perl’s pragmatic view to practicality and usefulness.
The most exciting change is the new smart match operator. It implements a new kind of comparison, the specifics of which are contextual based on the inputs to the operator. For example, to find if scalar $needle is in array @haystack, simply use the new ~~ operator:
if ( $needle ~~ @haystack ) ...
The result is that all comparisons now just Do The Right Thing, a hallmark of Perl programming. Building on the smart-match operator, Perl finally gets a switch statement, and it goes far beyond the kind of traditional switch statement found in languages like C, C++ and Java.
Regular expressions are now far more powerful. Programmers can now use named captures in regular expressions, rather than counting parentheses for positional captures. Perl 5.10 also supports recursive patterns, making many useful constructs, especially in parsing, now possible. Even with these new features, the regular expression engine has been tweaked, tuned and sped up in many cases.
Other improvements include state variables that allow variables to persist between calls to subroutines; user defined pragmata that allow users to write modules to influence the way Perl behaves; a defined-or operator; field hashes for inside-out objects and better error messages.
It’s not just language changes. The Perl interpreter itself is faster with a smaller memory footprint, and has several UTF-8 and threading improvements. The Perl installation is now relocatable, a blessing for systems administrators and operating system packagers. The source code is more portable, and of course many small bugs have been fixed along the way. It all adds up to the best Perl yet.
For a list of all changes in Perl 5.10, see Perl 5.10’s perldelta document included with the source distribution. For a gentler introduction of just the high points, the slides for Ricardo Signes’ Perl 5.10 For People Who Aren’t Totally Insane talk are well worth reading.
Don’t think that the Perl Porters are resting on their laurels. As Rafael Garcia-Suarez, the release manager for Perl 5.10, said: “I would like to thank every one of the Perl Porters for their efforts. I hope we’ll all be proud of what Perl is becoming, and ready to get back to the keyboard for 5.12.”
Perl is a standard feature in almost every operating system today except Windows. Users who don’t want to wait for their operating system vendor to release a package can dig into Perl 5.10 by downloading it from CPAN, the Comprehensive Perl Archive Network, at http://search.cpan.org/dist/perl/, or from the Perl home page at www.perl.org.
Windows users can also take advantage of the power of Perl by compiling a source distribution from CPAN, or downloading one of two easily installed binary distributions. Strawberry Perl is a community-built binary distribution for Windows, and ActiveState’s distribution is free but commercially-maintained. ActiveState’s distribution is available now, and Strawberry Perl’s is imminent.
The Perlbuzz section CPAN Watch is going on an indefinite hiatus. Kirrily Robert, Skud, who has driven CPAN Watch since its start, is moving to San Francisco from Australia, and just doesn't have the time to keep things going.
I'll still be keeping you informed about cool new CPAN releases, but instead of CPAN Watch I'll be putting them in the Mechanix section. So, point your RSS/Atom readers over there for now.
I placed the print order for 400 copies of The Vondish Ambassador this morning. Got the proof copy yesterday, and found it good.
This book is about the same format as the six Wildside Press reprints in the Ethshar series; I’m afraid it doesn’t match The Spriggan Mirror.
Anyway, the books should be here in mid-January. Sorry it couldn’t be in time for Christmas.
Very useful, thanks Andre!
I use Multi-Safari, and it's great: http://michelf.com/projects/multi-safari/
In 10.5.1 the older versions of Safari have problems, but a fixed copy of Safari 2.0.4 can be found here: http://tripledoubleyou.subtlegradient.com/stuff/Safari2/
I've just uploaded CPAN::FindDependencies version 2 to the CPAN. This is pretty much a complete re-write which has lots of cool new Stuff:Why am I telling you this? Cos, for the perl-qa folks, this will be the new brane for http://cpandeps.cantrell.org.uk/ soon. For the module-authors list - the bundled 'cpandeps' program is a useful tool for getting an idea of how much extra code gets pulled in when you make your users install a dependency. Re-using code is a Good Thing of course, but taken to extremes it can be a real pain in the arse for those for whom perl is just another environment that they have to support on their machines.
- uses Parse::CPAN::Packages instead of CPAN.pm, so loads faster and takes a lot less memory;
- much better core module detection;
- user can specify which version of perl to use when figuring out what's in core;
- can cache META.yml files and use a local 02packages file instead of fetching it from a CPAN mirror at startup;
- add 'maxdepth' parameter to limit how far down the dependency tree it goes;
- much better tests;
- 'cpandeps' script is now documented
I wrote about David's dependency checker before, and it's pretty slick.
Yann Kerherve has released Find::Lib, an alternate way of loading libraries in non-standard locations. Find::Lib lets you specify paths like so:
use base Find::Lib '../bootstrap/lib' => 'My::Bootstrap', %param;
I'd love to hear from anyone who's used it and what they think.
As I think most who have known me would have predicted, I stepped up to an iPhone several months ago. This post is authored from that device while I wait to get into the Dr.’s office with Ian. Entering text is certainly possible, though this is going to be a short post because its far from pleasant. Still, its a remarkable device that has reshaped the accessibility of information for me on many occasions. If it has one weakness, its the coupling with AT&T, whose service seems positively Neolithic compared to verizon…
More on this topic later. (Told you this would be short)
Peter wrote in and queried me about the new Doctor Who Series. He asked if someone new to the shows should start with Christopher Eccleston’s incarnation, OR just jump in at the beginning of David Tennant’s tenth Doctor. I figured I’d share my opinion here and give another plug for the new series, because it really is outstanding and a great deal of fun.
Definitely start with Christopher Eccleston!
There are a couple of good reasons for this actually, aside from the fact that he’s brilliant in the role and really brings out a very fresh, and somewhat tortured version of the Doctor to the audience. Tortured? Yeah, well you see… there was a war between the Daleks and the Timelords… simply called the Time War. A war so devastating that both sides seem to have been wiped out. So he’s dealing with a ton of survivor guilt, and is now alone in the universe. Which is fertile ground for the character to pick up some serious emotional depth, which Christopher Eccleston portrays very well.
Alot of the history of what happens in that first season also plays out in the 2nd and 3rd season under David Tennant, who is also brilliant, so starting at the new beginning helps make sense of a number of things down the line.
The 4th season starts in April of 2008 I believe, and then there will be at least a 1 year hiatus punctuated by 2 telefilms, and then the 5th season in 2010. On the one hand, that sucks, but it beats having the production crew and writers of the show quit or grow stale.
Here in the US, it’s Thanksgiving, a day of eating lots of food, watching football, and sometimes, just sometimes, expressing gratitude and giving thanks for those things that make life wonderful.
Here are the things I’m grateful for in late 2007, in no particular order after the first.
Google’s project hosting service has been a godsend. It’s changed the way I do open source projects. It has leapfrogged SourceForge for ease of maintenance, and the bug tracker trumps RT for CPAN that we’ve been using for so long. Add that to the integration with Google Groups which makes it trivial to create mailing lists, and it’s at the tops of my list for 2007. I can’t say enough good about it.
Eleven weeks ago, Skud and I started this little website called Perlbuzz as an alternative to the “more traditional outlets” for news in the Perl world. The response has been tremendous. We get 600 RSS readers every day, and have had over 10,000 unique visitors in that time. It makes me happy that our little venture is used and appreciated by the community.
It’s been over a year in the making, but the new version of the crucial Test::Harness 3.0 means more flexibility for module authors, and lots of UI improvements for people who just want to run prove and make test.
MJD is so much a fixture in Perl it’s easy to forget that he’s there. For 2007, though, never mind all the things he’s done for Perl in the past, or the hours I’ve spent being enthralled in talks of his. His Universe Of Discourse blog is the single most intelligent blog out there, and sometimes it just happens to be about Perl.
Was Andy Armstrong always around, or did I just not notice? His time and dedication spent on climbing on board with Ovid and Schwern and the rest of the Test::Harness 3.0 crew has been invaluable in getting it out. Plus, he’s a really swell guy anyway.
When I finally despaired of the amount of time and frustration it took to organize content for Chicago.pm’s Wheaton meetings, Dave Hoover stepped up and volunteered to take it over. I’m thankful, but not as much as I hope the other Chicago.pm folks are.
I’m all about having the machine keep an eye out for the stupid things we do, and the goodness of Perl::Critic is always impressive. You won’t like everything Perl::Critic says about your code, but that’s OK. It’s an entire framework for enforcing good Perl coding practices.
The Perl community is populated by some tremendous folks. Some names are more known than others, but these people help make daily Perl life better for me. In no particular order, I want to single out Pete Krawczyk, Kent Cowgill, Elliot Shank, Liz Cortell, Jason Crome, Yaakov Sloman, Michael Schwern, Andy Armstrong, Ricardo Signes, Julian Cash, Jim Thomason, chromatic, Chris Dolan, Adam Kennedy, Josh McAdams and of course Kirrily Robert. If you think you should be on this list, you’re probably right, and I just forgot.
Because even if she doesn’t understand this part of my life, she at least understands its importance to me.
I’d love to hear back from any readers about what they’re thankful for. I’m thinking about having a regular “Love Letters to Perl” column on Perlbuzz where people write about what they love in Perl.
I am typing this on my Apple keyboard, which is connected to my Asus Eee PC. The Eee PC is also connected to a old Apple VGA monitor running at 1152×864 (though there is a black border all around the screen.) I’m mainly testing the actual OS right now, which happens to be Xandros. It works pretty well.
Xandros is pretty nice. I know Ubuntu is the favorite OS of most desktop Linux people I know, but Xandros seems ok. I still need to customize the system a bit, but for now, it’s pretty functional. I’ve run Firefox, OpenOffice, Skype, ssh’d into a server, played with the webcam, and there is still much more to do.
First, my gripes… On the tiny keyboard there does not seem to be a caps lock indicator, so you don’t know when the caps lock is on. If you’re typing text you can see, it may not be a big deal, but typing a password you see **** and have no idea if it’s uppercase or lowercase. (Note: There is a caps lock indicator, but it’s on screen, I just didn’t notice it at first.) When I plugged in the external monitor it didn’t show immediately (it might be the Mac user in me that expected that) and I had to open a control panel to get it to show up. The trackpad button seemed like it took more pressure to click that it should. Maybe it will loosen up in time. I also need to get used to tapping on the trackpad to click, since I’ve never had a machine that supported that. (I may end up getting a small mouse to use with it.) I noticed the wife had used it to check her email (webmail) and I asked how she like it, her reply was “It was fine, expect for the small screen and hitting four keys when I wanted to hit one.” :)
The kids think it very cool. They’ve already asked for their own, for Xmas, or when they go to high school or college. At some point I’ll let them use it and see what they think. They use Macs and Windows PC’s pretty often, so they might have a different perspective.
Getting the wifi functional was a bit of work, but I will blame that on my network setup at home. (I’ve got a few old Macs connected via USB wifi adapters which complicate things.) I’m pretty sure the wifi is solid now, but we’ll see how it goes. (Update: One thing I noticed was that I wasn’t waiting long enough after booting for it to connect. It just took another minute or so for the connection to be made, then all was good.) When I disconnected the external monitor, I couldn’t use the built in screen because everything was off screen, as it was displaying the LCD display at the resolution I had the external monitor running at, so another note, be sure to set the display back properly for the internal LCD before you disconnect the external monitor.)
Did I mention it’s tiny? You know it’s tiny, but until you see one in person and use it, you don’t appreciate how tiny it is. It seems funny right now to be using such a tiny computer with a full sized keyboard and a 17″ monitor. I’ll be bringing it with me to the next Web414 Meeting so you can see it in person.
Here in the US, it's Thanksgiving, a day of eating lots of food, watching football, and sometimes, just sometimes, expressing gratitude and giving thanks for those things that make life wonderful.
Here are the things I'm grateful for in late 2007, in no particular order after the first.
Google CodeGoogle's project hosting service has been a godsend. It's changed the way I do open source projects. It has leapfrogged SourceForge for ease of maintenance, and the bug tracker trumps RT for CPAN that we've been using for so long. Add that to the integration with Google Groups which makes it trivial to create mailing lists, and it's at the tops of my list for 2007. I can't say enough good about it.
The readers of PerlbuzzEleven weeks ago, Skud and I started this little website called Perlbuzz as an alternative to the "more traditional outlets" for news in the Perl world. The response has been tremendous. We get 600 RSS readers every day, and have had over 10,000 unique visitors in that time. It makes me happy that our little venture is used and appreciated by the community.
Test::Harness 3.0It's been over a year in the making, but the new version of the crucial Test::Harness 3.0 means more flexibility for module authors, and lots of UI improvements for people who just want to run prove and make test.
Mark DominusMJD is so much a fixture in Perl it's easy to forget that he's there. For 2007, though, never mind all the things he's done for Perl in the past, or the hours I've spent being enthralled in talks of his. His Universe Of Discourse blog is the single most intelligent blog out there, and sometimes it just happens to be about Perl.
Andy ArmstrongWas Andy Armstrong always around, or did I just not notice? His time and dedication spent on climbing on board with Ovid and Schwern and the rest of the Test::Harness 3.0 crew has been invaluable in getting it out. Plus, he's a really swell guy anyway.
Dave HooverWhen I finally despaired of the amount of time and frustration it took to organize content for Chicago.pm's Wheaton meetings, Dave Hoover stepped up and volunteered to take it over. I'm thankful, but not as much as I hope the other Chicago.pm folks are.
Perl::CriticI'm all about having the machine keep an eye out for the stupid things we do, and the goodness of Perl::Critic is always impressive. You won't like everything Perl::Critic says about your code, but that's OK. It's an entire framework for enforcing good Perl coding practices.
The Perl Community in generalThe Perl community is populated by some tremendous folks. Some names are more known than others, but these people help make daily Perl life better for me. In no particular order, I want to single out Pete Krawczyk, Kent Cowgill, Elliot Shank, Liz Cortell, Jason Crome, Yaakov Sloman, Michael Schwern, Andy Armstrong, Ricardo Signes, Julian Cash, Jim Thomason, chromatic, Chris Dolan, Adam Kennedy, Josh McAdams and of course Kirrily Robert. If you think you should be on this list, you're probably right, and I just forgot.
My wife, Amy LesterBecause even if she doesn't understand this part of my life, she at least understands its importance to me.
I'd love to hear back from anyone about what they're thankful for. I'm thinking about having a regular Perlbuzz "Love Letters to Perl" column where people write about what they love in Perl.
The monthly Parrot release is out, version 0.5.0. Included in this version:
If you have or can install Inline::C, I'd greatly appreciate your help testing IO-CaptureOutput-1.05_53.
I've recently adopted IO::CaptureOutput, which is a wonderful tool for capturing program output to STDOUT or STDERR without ties and regardless of whether the output comes from perl, XS or programs in a subprocess.
However, the tests for XS use Inline::C and the C code was found to have portability problems (i.e. segfault on some Win32 platforms). At least one fix for Microsoft Visual C++ (MSVC) then broke on someone else's Linux platform.
(Aside: the fact that it it this hard to portably print the equivalent of "Hello World" to STDOUT and STDERR just astonishes me.)
My latest attempt at updating the C code now uses different code for MSVC and other compilers and now I want to test this as far and as wide as I can.
So if you have installed Inline::C and something that can send test reports (e.g. CPAN::Reporter), please test IO-CaptureOutput-1.05_53. For example, from the CPAN shell:
cpan> test DAGOLDEN/IO-CaptureOutput-1.05_53.tar.gz
Thank you very much,
-- dagolden
In the old software days with point releases, major versions would increase from 1 to 2 to 3, etc. Releases in between major versions would point releases along the lines of 1.1, 1.2, 1.3 and smaller releases would be 1.1.1, 1.1.2, 1.1.3, etc. Then came along Windows 95 and the exit of sequential version numbers. With this naming scheme you really can't have Windows 95.1 so we now have Releases, along the lines of Oracle 11g Release 1 and Windows 2003 Server Release 2. You can pretty much guarantee that there isn't going to be an Oracle 11.1g ;)
That's all fine and good from a marketing perspective if the reason is that we are now using a year or abbreviation instead of a simple integer but are there other technical reasons? I recently upgraded from Apache httpd 2.0.x to 2.2.x and the major thing that I encountered was that configuration had changed significantly and that I had to redo my conf files. I've spoken with some people that indicated many organizations are afraid of point releases for enterprise software because they often break things and are not necessarily smooth upgrades. This fit with my Apache httpd experience which got me thinking.
If there exist enough backward compatibility problems with point releases, it would make sense that software publishers would want to avoid point releases (at least from a marketing perspective), when the release is backward compatible, e.g. Releases for former point releases, Service Packs for aggregated patches and the like. Has the single point (vs. double point) release come to mean that backward compatibility has been broken. If so, should it be avoided from a marketing perspective when backward compatibility still exists?
I've been looking for a program that will take full screen shots of web pages even when the web page is larger than the window size on my physical screen, requiring scrolling. This morning I found such a program in Petr Šmejkal's Win32::CaptureIE when it was mentioned by Displeaser on DevShed Forums in the "Screenshot of webpage" thread. It uses ImageMagick for image manipulation.
From reading the Win32::CaptureIE POD, the CapturePage function does exactly what I want:
CapturePage ( ) Captures whole page currently loaded in the Internet Explorer window. Only the page content will be captured - no window, no scrollbars. If the page is smaller than the window only the occupied part of the window will be captured. If the page is longer (scrollbars are active) the function will capture the whole page step by step by scrolling the window content (in all directions) and will return a complete image of the page.
After installing ImageMagick, Image::Magick and Win32::CaptureIE on my Windows / ActiveState Perl system, I generated this screen shot with the following short program using no additional processing:
#!perl use strict; use warnings; use Win32::CaptureIE; StartIE( width => 900 ); Navigate( 'http://www.dev411.com/blog/' ); my $img = CapturePage(); $img->Write( 'capture.png' ); QuitIE;
Perl and CPAN continue to amaze me with their treasure trove of functionality. Are there similar tools for using Firefox, Linux, other image libraries or languages?
UPDATE: ishnid has found two programs with CLIs (posted to the same thread):
UPDATE 2:: I recently tried Win32::CaptureIE with ImageMagick 6.3.0 and it doesn't work. Apparently there used to be a link to "PerlMagick" in older versions of ImageMagick that may not exist anymore. Unfortunately Win32::CaptureIE relies on PerlMagick.
UPDATE 3:: I just tried the free version of Pearl Crescent with Firefox 1.5.0.7 which it says it should support but I get a "Download error" with pageserverbasic-1.3.xpi.
[…] I’m spending some time trying various tools like iChatAV (which I’ve mentioned previously), WebEx, and Skype (Download Skype For Windows). WebEx is the heavywe […]
There is a lot of choice on the CPAN for open source Perl libraries and sometimes it's difficult to get an idea of how modules compare to each other. CPAN Ratings is a good source of reviews but it's not convenient to compare one module with another. To provide a partial solution, I whipped up a quick CPAN Compare page which will pull the CPAN Ratings from a number of modules and summarize them for you.
I decided to use YUI DataTable for this. I've heard good things about YUI so I decided to give it a try. Getting the example code to work off of the Yahoo website almost as straight forward as say using Scriptaculous demos but it was faster than working with Dojo in the early days. The nice thing about the DataTable is that it takes a JavaScript array which can be populated using server-side JSON generated code. I used JSON::XS for this.
YUI DataTable has a nice sorting feature and it can sort on text, numbers, dates, etc. However, it does not seem to be able to sort on visual information only so if you include HTML markup, that will be used for sorting as well. To get around this I used standard text sorting and customized the title fields to assist in the sorting. For example, in a link, I start with <a title=" instead of <a href=" because title is arbitrary and can be used to mirror the InnerHTML. For numbers a text sort will have 25 come before 4 so I added leading zeros to numbers using sprintf and put them in the title attribute as well.
A few Perl modules and the Logo Creator website made this easy to set up. YUI DataTable has a nice default CSS so I just left that as is.
Note: YUI DataTable is convenient if you just drop in a Perl data structure and have it generate the HTML and JS for you. This script uses 3 DataTables (ratings, popular and recent) so I wrote a Perl wrapper for YUI which takes a hashref and generates the client code, extracting the fields from the column definitions. This works because YUI does not require the HTML table to be built beforehand. By not having an underlying table, it's faster to get running but also won't fallback as nicely for people who aren't running JS (8% of users?). As an alternative, jQuery has a couple of add ons which work by enhancing an existing HTML table. jQuery has some nice syntax but I haven't gotten around to using it yet. Perhaps it's worth a look.
ack, my replacement for grep for 95% of the times programmers use grep, just got released to CPAN with version 1.70.
At long last, you can now get contextual lines before and after matched lines, just like GNU grep's -A, -B and -C options. You can also match on a specific line number or range of line numbers with the new --line option. For example, if you want to see the first line of every Perl file in a tree, you'd just do ack --line=1 --perl. Thanks very much to Torsten Biix for putting both these features together for me.
Finally, Elliot Shank pointed out that one of my favorite features, the -1 option, was never documented. Now it is. The -1 option says "stop after the first match of any type." If you find yourself acking for lines, or searching for a specific file with ack -g and then having to Ctrl-C to stop the search process, just add a -1 and Ctrl-C no longer.
ack is available in the ack distribution on CPAN, or by installing the module App::Ack from the CPAN shell. You can also download the single-file version direct from Subversion and drop it right into your ~/bin directory.
Glenn Loos-Austin |
No longer do you have to do a $mech->save_content() and then run mech-dump on it. How has it taken me so long to put this stuff in there?$agent->content_contains( qr/\QEnter keyword(s)/ )
or $agent->dump_all( \*STDERR );
not ok 14 - Content contains '(?-xism:Enter\ keyword\(s\))'
# Failed test 'Content contains '(?-xism:Enter\ keyword\(s\))''
# at t/simple-search.t line 31.
# searched: "<HTML>\x{0a}<HEAD>\x{0a}<TITLE>TitleTales™</TITLE></HEA"...
# can't find: "(?-xism:Enter\ keyword\(s\))"
/buttonsd/bisac2.gif
/graphics/bar.gif
POST http://hoops.flr.follett.com:2112/simpsearch.php [simsearch]
clickval= (hidden readonly)
searchwords= (text)
S=<UNDEF> (checkbox) [*<UNDEF>/off|on/Include Out of Print / Please Order Direct Titles]
This weekend has been a sporting weekend of woe. England lost in the Rugby World Cup Final, although of course I'm glad that we made it that far. Young Mr. Hamilton failed to capture the drivers world championship in his first season, and Forest only managed a draw at home against Doncaster.
Still, I guess Orient are once again winning, which is good.
And on the bright side, the Montreal Canadiens recorded their first home win of the season. And the New England Patriots continued their unbeaten season with a blow-out at Miami.
Saturday Katrien and I were in Mont Tremblant for much of the day, trying to enjoy what will be our last weekend together for a while – Katrien will back on the European side of the pond for about six weeks, which is going to feel quite strange. Her Visa has not yet come through, and until it does she can only spend six months out of the year here. We want to make sure she can be here at Christmas, and that means spending some time out of the country now. She's looking forward to being in London, but it will seem strange.
I was on my way to visit a client on Friday morning, just about getting caught in the morning rush hour traffic, and I started to develop a headache.
It kept getting worse and worse, and the next thing I knew I started feeling nauseous. "This isn't good" I thought looking for an exit, somewhere. It kept on getting worse and worse, and just as my vision was starting to blur, I got to an exit. I pulled off the freeway and into a gas station, where I vomited.
"Shit" I thought. I can't visit a client like this. It must be something I ate. I was really feeling terrible at this point. My hands and feet were starting to feel numb, I was an hour from home, and I didn't know what to do.
Nothing else for it, I need to get home.
That's pretty much the last thing I remember from Friday morning.
The next thing I really knew, I woke up in bed at home. Apparently I had called the people I was supposed to be meeting and canceled, called home and said I was on my way, driven back, stumbled into the house, very lucidly told Katrien what was wrong, and then fallen asleep mid-sentence.
This of course, was rather worrying.
It turns out that I'm suffering from a Post-Dural Puncture Headache.
I had an epidural to try to help my back on Thursday, and it didn't go very well. What has happened is that the sack that contains the brain, spinal nerves, and all that goodness (known as the "dura") got punctured by the needle while the epidural procedure was being performed. The Cerebral-Spinal Fluid that surrounds the brain is leaking out of the sack through this hole, and that gives you a headache. When you are upright (either sitting or standing) it gets worse and worse, because of course the fluid that is remaining in the sack goes to the bottom, leaving a nice big air bubble around your brain where liquid is supposed to be…
So I'm stuck lying down. As long as I'm lying down it doesn't hurt too much, but as soon as I stand up it's like I've got the worst hangover ever.
All the pain of getting drunk, but none of the pleasure.
Apparently you can get a "blood patch" to help heal the puncture faster however, this involves taking blood and then injecting it above the puncture so that it forms a clot. I'm not sure that I'm too keen on that to be honest.
Right now I'm fed up of doctors, medicine, and anything to do with my insides.
I've just finished watching Battlestar Galactica Season 3, and now I need to know what happens in season four. Of course it doesn't air for another 5 weeks or so, but I'm genuinely curious, and will probably make the effort to watch it on TV provided that I know what channel it airs on here!
Does anybody know where to look for BSG in Canada? Help! I can't wait until the DVD release in a years time!
As of Perl::Critic 1.07, I would like to discourage the creation of constructors for Policies. Instead, I would encourage the use of P::C::Policy::initialize_if_enabled(). The reasoning is twofold.
One, this allows initialization to be deferred to the point where we know that the policy is going to be used. P::C::PolicyFactory always instantiates every Policy that it can find. It is up to the P::C::Config object to filter that set down. Primarily, this is an issue for Policies which dynamically load other modules.
Two, this method enables the Policy to decide for itself whether it should be enabled. This means that a Policy that depends upon a module that might not be present can remove itself from the set that violates() gets called on, thus speeding things up because it isn't being called on every PPI::Element.
This originated from Chris Dolan's work on the Perl Foundation grant to create the remaining Policies that can be implemented that enforce one of the ideas in PBP. Specifically, for Documentation::PodSpelling, but this change has been made to all the configurable core Policies. In particular, this helps CodeLayout::RequireTidyCode.
Differences from a constructor other than the obvious first parameter:
This is how the two above policies bow out.
They don't get along.
As of the recent 1.07 release, Perl::Critic
, has started using Readonly
to be more more self-compliant with Perl Best Practices. We had been avoiding use of constant
for the reasons described in the book, but had not been willing to add the Readonly
dependency until now.
The Perl::Critic
coding standard has been to use sigils for subroutines in @EXPORT_OK
, etc. and import lists, but Exporter
treats them as optional. And, in fact, there's code that strips them off (line 47 in v5.58). I haven't figured out the commonality, but in a few environments, this fails spectacularly. Once we removed the subroutine sigils from everywhere, the problems have gone.
Explicitness: 0, Keyboard laziness: 1.
I just recently picked up a digital voice recorder (aka DVR, not to be confused with digital video recorders) for recording conference calls and meetings. In three short meetings I have become a true believer. I always taken detailed meeting notes but that was because I would write notes during the meeting. With a DVR, I can concentrate on running the call and going back to catch the details later.
For my first DVR I picked up the Olympus DS-30 from FRYs. The benefits that I keyed in on where the large-looking stereo speakers and the noise reduction. Since this is my first DVR I was easily impressed by the utility of it. So far I've recorded and played back on the device, copied the WMA files off using it as a USB storage device on Win XP and converted the WMA to OGG Vorbis using dbPowerAmp. The only thing that doesn't seem to work is the CD that it came with. XP would not recognize it at all but at least I don't need since it doubles as a USB device.
Although it meets my current notetaking requirements easily, I've been thinking about whether it'd be good to use for recording podcasts. My current issue is that it records in WMA and not a FOSS standard. After looking over a number of DVRs, it seems that the higher end ones use WMA, LPEC, DSS, etc. but not common music formats such as MP3 and OGG. What native format do you think is the best for DVRs? Is it fine to record as WMA and convert to OGG Vorbis or are there better options?
I don't know too much about voice recorders at the moment so I'm easy to please. Which ones do you like and what are important features for you?
I've worked on a number of database-driven projects and no matter how much people want database abstraction, it was always difficult to code and maintain. I was recently reminded of this when I read this Drupal article on dropping PostgreSQL support. Not only can it be difficult to maintain support for multiple databases, but it may be difficult to find developers.
One solution of modern programming is to move database abstraction from the code to the infrastructure using a ORM (Object-Relational Mapper) or Data Mapper. A ORM and Data Mapper abstracts the database for you so you no longer have to do tie db abstraction to each app. Not only does it let you code once for multiple databases it lets your users migrate their data from one database to another. This blog runs Typo which is based on Ruby on Rails and ActiveRecord. I've been contemplating migrating Typo from MySQL to PostgreSQL and I've been told that it would be as simple as exporting the data with YAML, updating the database.yml file and importing the data. I haven't gotten around to doing it yet but it is a powerful idea. ActiveRecord is a data mapper and isn't as flexible as a full blown ORM but it gets the job done for the most part. For a full-blown ORM, I think of Perl's DBIx::Class which provides a full OO interface to the RDBMS allowing you to code just once for multiple DBs without limiting you when you want to use some esoteric database-specific SQL. DBIx::Class is often used with the Catalyst Framework but is also used by itself.
There are PHP frameworks out there like Symfony and Cake but do any of them have stand-alone ORMs? If so, could Drupal move to something like that and solve their maintainership problems once and for all? Drupal is part of the Go PHP5 effort so there should be no issue using PHP 5 OO. Something to think about for the Drupal folks if a PHP ORM is available.