Pensieri di un lunatico minore » Programming

Emacs macro trick

petrilli — Mon, 07 Sep 2009 16:28:52 +0000

I don’t know when this was added, but as someone who has used Emacs pretty constantly since 1985, I can’t believe I never knew this. Most people know you can record a simple macro with C-x ( to start and C-x ) to end. You can then replay it back with C-x e. Nothing new there, right? Well, obviously you can combine the C-u prefix, which will repeat it. So C-u 10 C-x e will execute the macro 10 times. If you say C-u 0 before, then it’ll run until it has an error (usually the end of the file, or something).

But here’s the trick. If you type C-u C-x e, it will run the macro once, and then allow you simply to press e to keep running it. This is very handy when you’re trying to go through a list in the middle of a document.

Funny how I never noticed.

The Ring of Perl

petrilli — Thu, 04 Dec 2008 15:15:32 +0000

I see a post entitled Perl 5 Programmers Are Dying, and all I can think of is The Ring. Odd, eh?

Obama campaign needs talented geeks

petrilli — Sun, 28 Sep 2008 00:17:33 +0000

If you’ve got talent and time on your hands:

The Obama campaign’s grassroots movement is supported by the best data management in politics.

The Obama Data team is looking for data-savvy supporters to join our team as Data Fellows for the last weeks of the campaign. The position is full time through November 4th.

Data Fellows will be required to attend an online training and will be expected to volunteer full-time in the state of their placement for three weeks. Data Fellows will also:

Assist the voter file manager and data team with data processing, data compilation, or reporting needs

Assist the organizers with data-intensive tasks and projects

Help build this movement in any way they can

Apple online.

Speed freaks

petrilli — Mon, 21 Jul 2008 01:19:24 +0000

Other’s muse on the absurdity of those obsessed with speed. Like a meth addict, the use any possible rationale to justify their unhealthy obsession. My favorite is:

“What do you mean my algorithm runs at O(n!)? It’s twice as fast as yours with this small test set”

The number of times I’ve seen someone rant about how fast their new algorithm is—when running on an irrationally small sample—is uncountable. As I’ve said a million times:

Make it work
Make it right
Make it fast enough

Profit!
Seriously. Don’t put things in the wrong order, or you’ll never get anywhere.

More Mosquito Lisp

petrilli — Fri, 06 Jun 2008 13:55:26 +0000

Scott Dunlop writes about the transition of his work from Mosquito Lisp (MOSVM) to Wasp Lisp. One of the thing that I wanted to point out about my interest, though, was that it wasn’t strictly about it being a Lisp dialect. It’s more what that brings with it: flexibility. It’s a tight, very small, and reasonable performing implementation that’s very flexible. It also comes with, to steal a phrase from Python, “batteries included”. This makes developing neat applications nearly trivial.

Close, but note quite close enough

petrilli — Wed, 04 Jun 2008 21:03:02 +0000

So today I was trying to debug some code, and for some reason, I couldn’t see what was wrong for a few seconds. Then, I realized that I had been typing lambada instead of lambda. Time to stop coding for the day.

Uninformed commentary

petrilli — Mon, 02 Jun 2008 20:59:02 +0000

One of the great things about the intertubes is that it allows people of all capabilities and skill sets to shoot off their mouth and make fools of themselves. Recently, Sho Fukamachi demonstrated the truly epic capability to not only miss the entire point, but demonstrate a nearly bottomless lack of knowledge of the area of virtual machine technology and what actually represents “state of the art”. Oddly, it reminds me of me when I was 20. Fortunately, Patrick Collins straightens it out. Oh, and I’m quite aware of object databases in the public eye that are nearly 1PB compressed. There are some in the intel community that even larger and more complicated.

It’s amazing how some communities think that they’ve actually had an original idea. There are almost no original ideas left. The only reason anyone thinks their “idea” is original is that they’re simply oblivious to the decades of work of others.

Sacred cows

petrilli — Sun, 01 Jun 2008 17:19:53 +0000

It’s amusing as all hell to watch some people blow up about the MagLev Ruby implementation. Personally, while I find Ruby interesting, the performance is embarrassingly bad for a lot of problem domains. That doesn’t mean it’s not useful, it just means it’s not as useful as it could be. I’d like to take a couple comments that Charles Nutter made and just “observe”:

First off, they demonstrated its distributed object database automatically synchronizing globally-reachable state across multiple VMs. It’s an amazing new idea that the world has never really seen…

except that it isn’t. This is based on existing OODB technology that Gemstone and others have been promoting for better than a decade. It’s cool stuff, no doubt, but it’s been available in Gemstone’s Smalltalk product and in their Java product for years, and hasn’t seen widespread adoption. Maybe it’s on the rise, I really don’t know. It’s certainly cool, but it’s certainly not new.

There are only a tiny handful of companies in the world who can show production object environments of that scale. Gemstone and Objectivity being two of the most successful. Both are rooted in the Smalltalk world originally, though offer other products to most of their customers. What I find troubling is the dismissive nature that Mr. Nutter has about the problem domain. The fact that most people worship at the relational temple doesn’t mean that it’s actually a good idea in many cases. Some of the most complex databases in the world are object databases. There’s a reason a huge percentage of Objectivity’s money comes from the intelligence community. Data fusion is everything.

Except that these are results reported entirely in a vacuum. Whether this is fib following the “rules” of Ruby is entirely an open question. Whether this is method dispatch adhering to Ruby’s call logic is entirely an open question. Whether this is a while loop using all method calls for its condition and increment steps is an open quesetion. Because the Maglev guys haven’t started running Ruby tests yet. Is it Ruby?

Well, once someone actually defines what Ruby is we can have that discussion. One of the most frustrating aspects of Ruby is that there is no actual language definition. There is only a language implementation, several at that, and if I recall correctly all have had to reverse “engineer” the language from the implementation. This makes discussing “is it Ruby” a laughable prospect, and Mr. Nutter should know that. Let’s examine “some words of his from just a few weeks back”:

Compatibility is hard. I’m not talking a little hard, I’m talking monumentally hard. Ruby is a very flexible, complicated language to implement, and it ships with a number of very flexible, complicated core class implementations. Very little exists in the way of specifications and test kits, so what we’ve done with JRuby we’ve done by stitching together every suite we could find. And after all this time, we still have known bugs and weekly reports of minor incompatibilities. I don’t think an alternative implementation can ever truly become “compatible” as much as “more compatible”. We’re certainly the most compatible alternative impl, and even now we’ve got our hands full fixing bugs. Then there’s Ruby 1.9 support, coming up probably in JRuby 1.2ish. Another adventure.

So before the stones are cast, perhaps the house should be repaired?

Then there’s Maglev. Like the other impls, I’m excited that there’s a new possibility for Ruby to succeed. A high performance, “scalable” Ruby implementation is certainly what this community needs. But unlike most of the other implementations, it seems like Maglev is pushing performance numbers without compatibility metrics; marketing before reality. Am I far off here?

Here’s the thing. Maglev is a new language on top of an existing VM. One that has over a decade of hardcore production use, something that even Java can’t really claim. There’s a wide gulf between putting a new syntax on top of an existing VM architecture and building a new VM. VMs are intensely hard, and only a few people in the world are truly good at them. Dynamic VMs all derive from Smalltalk, and even Java is having to glue on all sorts of silly doo-dads to try and pretend it can support it without huge cost. Ruby is very much Smalltalk in different clothing, and therefore I would expect Ruby to run at near Smalltalk speeds once you clean up some pieces. I know the great wizard had a Python implementation running on a Smalltalk VM, alas it never got released.

The truth is that not all of these optimizations are kosher right now. Removing the ability to override Fixnum#+ certainly makes it easier to optimize addition, but it’s not in the spirit of Ruby. Removing frames may be legal in some cases (like this one) but it’s not legal in all cases. And of course I’ve blogged about how Thread#kill and Thread#raise are broken, but we have to support them anyway. On and on we can go through lots of optimizations you might make in the first 100 days of your implementation, only to back out later when you realize you’re actually breaking features people depend on.

Features, or design errors? Because if it’s the later, then breaking them is the best thing that can be done. The refusal to fix them in the past is simply a weakness. The totem of backward compatibility is what gave us Windows Vista.

Emacs color themes

petrilli — Fri, 16 May 2008 14:38:21 +0000

In my continuing adventures getting back to using Emacs, I was working on tweaking the color theme that I use. While it’s based on someone else’s originally, I’ve made quite a few minor changes. Somewhere along the line, I messed it up and started getting all sorts of strange errors that made no sense. They couldn’t possibly be wrong.

Here’s the old one:

(defun color-theme-charcoal-personal ()
  (interactive)
  (color-theme-install
   '(color-theme-charcoal-gray-personal
     ((background-color . "Grey15")
     ...

And here it is, fixed:

(defun color-theme-charcoal-personal ()
  (interactive)
  (color-theme-install
   '(color-theme-charcoal-personal
     ((background-color . "Grey15")
     ...

Do you see the difference? No? Look carefully.

It seems that if you name the defun differently than the first element of the first list passed to color-theme-install, it goes wonky. Unfortunately, the debugger wasn’t much help, and it was simply by stepping through a comparison with the original to see if somehow I’d missed something that I found the place where I’d screwed up. Something annoyingly tiny, and a mistake that should have generated a more useful error.

If you want the Elisp code, you can find it here: color-theme-charcoal-personal.el

Fun with Pymacs

petrilli — Mon, 12 May 2008 15:01:04 +0000

Pymacs was developed by FranÃ§ois Pinard, and it allows two-way communication between Emacs and Python. Very cool, except… it breaks my copy of Aquamacs, causing all sorts of things to spit out this lovely error:

Variable binding depth exceeds max-specpdl-size

What, you ask, is causing that? Well, it seems to be tied up with the advice framework (e.g., defadvice) and creating more recursion than Emacs will allow. Specifically, it trips at 600 levels. Very annoying.

The debugger is your friend; even in Java

petrilli — Wed, 07 May 2008 00:27:52 +0000

I am continually fascinated by how little many people use a debugger¹ when they’re trying to understand their code. Today, I watched an otherwise very intelligent person stare at some Java code for 15-20 minutes trying, apparently, to mind-meld with the JVM and understand what went wrong and why he was getting a glorious NullPointerException, something I still find gloriously amusing.

“Why don’t you set a breakpoint and look at what’s going on right there?” I asked. You would think I was talking Aramaic. Even I, arch-hater of Java—and Eclipse—was able to divine the way to do this task and within 15 seconds we had a solution.

Learn to use the debugger, even if you’re working in a dead language like Java. It might not be a live object universe like Smalltalk, but it is 100x better than simply guessing.

¹ If you’re cursed with either C or C++, invest your hard-earned money in TotalView if you have to do anything with multi-threaded code. It is unparalleled by any other debugger.

Writing code is easy; designing software is hard

petrilli — Wed, 07 May 2008 00:23:01 +0000

In reading an article about SciPy, the author has this gem:

The bottleneck in writing code isnâ€™t in the writing of the code, itâ€™s in understanding and conceptualising what needs to be done.

I mean gem quite literally. This, in the end, is the core difficulty that people seem not to grasp about writing good software. Writing lots of code is trivial; any monkey, or IDE can do it. The difficulty comes in the concept and more importantly the domain knowledge around the problem. For me, that’s 90% of the problem.

This is why I say languages don’t matter that much. Knowledge is language-insensitive.

The ad

petrilli — Sun, 09 Mar 2008 18:34:23 +0000

When I was in TX, I started seeing Senator Clinton’s infamous homage to LBJ: the telephone ad. I almost fell out of my chain, though, as she hasn’t got an ounce of applicable experience to make that decision, which puts her exactly where Senator Obama and Senator McCain are. None have ever been the executive. Dick Morris, never a retiring wall-flower, has some advice for the campaign:

The next time Hillary uses the recycled red phone ad, counter with one of your own. When the phone rings in the middle of the night, have a womanâ€™s voice, with a flat Midwestern accent, answer it and say, “Hold on” into the receiver. Then she should shout, “Bill! Itâ€™s for you!”

Because with Hillaryâ€™s complete lack of any meaningful experience in foreign affairs, and her lack of the “testing” that she boldly claims, sheâ€™ll be yelling for Bill.

I don’t think he should run that ad, as some might see it as misogynistic, even though it’s not. But at the end of the day, the only test she’s had, on the Iraq war, she freaked and picked the wrong answer.

Amazon Simple Queue Service

petrilli — Wed, 06 Feb 2008 01:56:19 +0000

Amazon just released a new API and pricing model for their Simple Queue Service. There are two changes that I find especially interesting:

Data transfer to EC2 is now free. You only pay for transfers to a system outside Amazon’s cloud. This makes it dirt cheap to use for many applications.

Pricing is now $0.01/10,000 requests, rather than $0.10/1,000 messages. It’s both a change in how things are measured, but also potentially a lot cheaper depending on your application. This means that you can run 1,000,000 requests for a single dollar. Not bad.
Overall, it’s a great change, I think. They have reduced the size of the maximum message from 256kB to 8kB, but I doubt this will affect many people, and you can just shove the larger messages into S3 and send a reference in the message.

Mercurial tips

petrilli — Thu, 06 Dec 2007 19:00:01 +0000

So, I’ve switched to using Mercurial for all my version control needs. As a distributed design, it makes it easy to work on a detached machine—say on an airplane—and not lose the functionality of a full version control system. This is in stark contrast to either CVS or even Subversion, which was my previous choice. All is not perfect, of course, but I’ve learned a few tricks that weren’t obvious, at least not to me.

First, Mercurial doesn’t track directories, at least not as such. This is a big change from Subversion. This means that some of my scripts, which set up a uniform directory layout for projects and build that as the skeleton in the repository, no longer work. I fixed this relatively easily by simply putting a file names 00README in the directory which contains a one line description of what the directory is for. Overkill, but it feeds my habits.

Second, I found it a bit annoying when I needed to “push changes”: that I had to specify the location every time. Turns out, you don’t have to, as you can create aliases and you can also create defaults. If, for example, you put the following in the .../.hg/hgrc file in the repository, you just use the default all the time:

[paths]
default-push =

Now you can just use hg push to publish your changes, and not have to do anything else. Much nicer.

Lastly, because it’s so easy to create new changesets, and takes almost zero time, I find myself checking things in much more frequently. This means that the changeset is smaller, and therefore easier to find issues should they arise.

Distributed Version Control

petrilli — Mon, 03 Dec 2007 18:56:03 +0000

Jeffrey Shell writes about moving to distributed version control:

The more I play with the new breed of VCS tools, the more I appreciate them. The older generations (CVS, SVN) look increasingly archaic, supporting a computing and development model that seems unsustainable. Yet most of us lived with those tools, or something similar, for most of our development-focused lives.

When I speak of the new breed, the two standouts (to me) are Git and Mercurial. There are some other interesting ones, particularly Darcs, but Git and Mercurial seem to have the most steam and seem fairly grounded and stable. Between those two, I still find myself preferring Git. Iâ€™ve had some nasty webs to untangle and Git has provided me with the best resources to untangle them.

While Jeffrey is using Git, I’m using Mercurial, and I have to agree with him. I see two distinct advantages to distributed VCS:

Ability to work totally disconnected

Light-weight branching, etc.
Jeffrey talks a lot about the light-weight nature of things. I can’t even begin to explain how critical the ability work totally disconnected is to how I operate. I spend a lot of time on airplanes—measured in hours-per-week—and the inability to check-in, merge, diff, etc., is a huge impediment to getting work done. Right now, I’m struggling with getting Mercurial to work properly under Windows with SSH as the transport protocol, but I’ve managed to work around it for now.

If you’re doing a lot of work, even if it’s just a single developer, you should definitely be checking it out.

Baroqueness

petrilli — Thu, 16 Aug 2007 02:52:07 +0000

Makefiles are evil; automake is a baroque disaster, likely understood by nobody currently living.

Testing and behavior in development

petrilli — Mon, 16 Oct 2006 17:50:17 +0000

I’ve been practicing test driven development on some percentage of my projects for a while now. Something, however, has always felt just a little bit tiring about the whole thing. The biggest thing I struggled with was what to test, and how best to test. I suppose that’s obvious, eh? My biggest failing was that I would often test things that don’t need testing; things like other people’s heavily tested frameworks. This made writing tests boring.

Then, I read Dave Astel’s posting on behavior driven development, and a lightbulb—dim though it may be—went off in my head. The problem with TDD isn’t the idea, but the perspective. Dave makes it clear in the following excerpt.

It’s about figuring out what you are trying to do before you run off half-cocked to try to do it. You write a specification that nails down a small aspect of behaviour in a concise, unambiguous, and executable form. It’s that simple. Does that mean you write tests? No. It means you write specifications of what your code will have to do. It means you specify the behaviour of your code ahead of time. But not far ahead of time. In fact, just before you write the code is best because that’s when you have as much information at hand as you will up to that point. Like well done TDD, you work in tiny increments… specifying one small aspect of behaviour at a time, then implementing it.

When you realize that it’s all about specifying behaviour and not writing tests, your point of view shifts. Suddenly the idea of having a Test class for each of your production classes is rediculously limiting. And the thought of testing each of your methods with its own test method (in a 1-1 relationship) will have you rolling on the floor laughing.

Eureka! Behavior Driven Development. Really what was missing was the context of everything. So, I went searching the treasure-trove that is Google, and came across an article by Luke Redpath that put it in a Ruby on Rails perspective. Specifically, the behavior of Models and how to specify them. That then led me to RSpec, which implements a DSL for writing specifications that is on top of the normal Test::Unit framework. There’s also a Ruby plugin that helps make things work smoothly.

Now, if only ZenTest worked with Specs. The auto_test piece would be very useful.

Updated (21 Oct 2006): Nick Sieger has done this already, apparently using his magical time machine.

A brilliant comment system for WordPress

petrilli — Mon, 09 Oct 2006 23:28:14 +0000

I’ve been using WordPress for a while, and generally am quite happy with it. One thing is is slightly weak on is the comment system. I thought that what I wanted was a hierarchical system—where you can reply to other comments—but after seeing Jack Slocum’s absolutely brilliant work, I want his system.

You really should see it to understand how great it is. It also shows off the Yahoo UI Javascript libraries.

Software Engineering

petrilli — Sun, 03 Sep 2006 00:43:19 +0000

Without comment, from Andy Hunt, comes software engineering in 10 panels:

Indeed.