Community Matters

Ok, this isn’t brand new, but I just found it… and haven’t even had the time to try it. But I have to give it extra points for coolness.

Dan at UneasySilence has figured out how to run Mac OS X on an eeePC. So after the default Xandros Linux, Fedora Linux, Debian, OpenSuSE, Ubuntu and of course Windows XP and even (Yikes!) Vista, the list of OSs to run on the eeePC is pretty complete now.

I’m still using the default Xandros install and Fedora, installed on an 8GB SD card, and switch between them via the BIOS… it’s still impressive that it’s possible :-)

Thanks for visiting!
I hope this was helpful - if not, please leave a comment and let me know why! Were you searching for something else? Did I miss an important aspect?

Given the mess that SCO is in (due to their own doing) and the mess they have created for the Linux community, I keep lose tabs on what they are doing. So I was thrilled to see another sign of the speed of innovation over there. The top news item on their home page doesn’t talk about their failed lawsuits or their Chapter 11 filing - no, it’s about a FREE (their caps, not mine) update to SCO Unix that fixes the daylight savings time issue. Great thinking and innovating, guys. One minor nit: that happened LAST year (my caps, not theirs).

I’ve been playing with a cool Wordpress plugin. AJAX’d Wordpress allows you to add a number of features to your site - what I really like was the ability to do in-line comments and commenting.

Pretty slick…

As a strong believer in open source I might be a bit biased here, but there’s one thing that puzzles me. Why aren’t companies more eager to have people innovate on top of their products?

Look for example at Intel. Back in 1990 they published complete instructions how to program the 80386 processor and accompanying chipset. The result? A young Finish student started to play with his new system and started a little project that went on to change the world of computing - Linux. Today Intel is still the most open and active supporter of open source out there (and yes, I might be even less objective here, given that I work for Intel - on the other hand it means that I know exactly what I am talking about when I say this).

Understanding the huge success that was caused by this openness, I am completely stunned to see other companies go out of their way to prevent people from innovating on their platform. Take Motorola. The Ming is a great phone, but there is no working SDK. Or even worse the iPhone. Where Apple is actively trying to prevent people from running third party software on the phone, to the extend of bricking phones that had non-Apple software installed on them.

That has two consequences. First, many innovators (and, btw, potential customers) are scared away and will simply not buy the product. That seems dumb. And second, the most enthusiastic people who aren’t scared off by these tactics are now wasting their time hacking the firmware over and over again (and they are succeeding, mind you… regardless how many smart people are working for a company, there will always be more smart people outside that company). These smart, enthusiastic people could instead be spending their time on creating the next killer application - which likely would even more increase the market for the product.

As I said, I don’t get it.

As usual, going to a good conference gets me excited again about the things happening in the community. I spent a few hours yesterday with the OpenID folks. What an amazing idea. I of course immediately created one - and then learned the hard way that support for them is still spotty (LiveJournal claims to support them, for example, but you can’t use an OpenID based login to comment on a post - there’s room for improvement…).

Also spent lots of times with the folks from Canonical and Ubuntu. I have to admit that I’m impressed. With their vision, their technology, but also their people. Pretty darn impressive.

And of course being around so many tech bloggers made me want to blog more about technology again. We’ll see how long it lasts, but right now I want to revive this blog…

Five weeks ago I decried that there is no open source project for the Ming. It turns out I was wrong! Johann pointed out in a comment that there’s OpenEZX, a project that is working on several of the Motorola Linux phones, including the Ming.

I haven’t spent enough time looking at what is there, yet, but it all looks mighty promising. Stay tuned, in my infinite spare time I certainly will play with this.

I wrote about setting up Postfix on Tiger before. But after quite a while of procrastination I decided I also wanted to do something about the flood of spam that was sent to hohndel.org. SpamAssassin seems to be the preferred method to go (if you are in the open source camp). It’s bundled with Mac OS X server - but why spend that money… it’s easy enough to set up from scratch.

These instructions are based on a posting by Kalinga Athulathmudali where he describes a similar setup, but not for OS X.

First make sure you have your CPAN setup straight. Some hints to make sure all is well are here.

Next, install SpamAssasin.
$ sudo perl -MCPAN -e shell
cpan[1]> install Mail::SpamAssassin
quit

Use the System Preferences of OS X to create a user named spamfilter. Give it a random password and make sure the user isn’t allowed to administer the system.

To work around a couple of issues with the way postfix deals with the return values of filters we’ll create a little script that will do the filtering.

# Clean up when done or when aborting.
trap "rm -f /tmp/out.$$" 0 1 2 3 15
# Pipe message to spamc
cat | /usr/bin/spamc -u spamfilter > /tmp/out.$$
/usr/sbin/sendmail -i "$@" < /tmp/out.$$
# Postfix returns the exit status of the Postfix sendmail command.
exit $?

Next we make sure that spamd is started whenever the system boots. For this we simply create an entry in the StartupItems - a tar file with the necessary instructions can be found on the SpamAssassin Wiki.

Now you need to make sure that SpamAssassin is called from postfix; edit /etc/postfix/master.cf with the following two changes. First make sure that smtp over inet looks like this

smtp inet n - n - - smtpd
    -o content_filter=spamfilter:dummy

and then add an entry for the spamfilter:

spamfilter unix - n n - - pipe
    flags=Rq user=spamfilter argv=/usr/local/bin/spamfilter -f ${sender} -- ${recipient}

the last line starting with flags=Rq is actually one line; this WordPress theme makes it a little hard to render this correctly.

Run postfix reload to force postfix to read the new configuration and watch your logfiles to make sure that spamd is called correctly (remember that you need to start spamd as root in the background - or you can just reboot which will take care of that as well). Your /var/log/mail.log file should contain entries like this:

spamd[???]: spamd: connection from localhost [127.0.0.1] at port 49671
spamd[???]: spamd: setuid to spamfilter succeeded
spamd[???]: spamd: processing message for spamfilter:???

The final step is now to filter the spam mail from your normal mail flow. I prefer to use procmail for that. A simple entry like this in your .procmailrc file should do the trick (but that depends on the folder layout of your preferred mail client… as I mentioned before, I prefer mutt. In that case this should work:

PATH=/usr/bin:/bin:/usr/local/bin:.
SPAMMAIL=$HOME/Mail/spam

:0:
* ^X-Spam-Level: \*\*\*\*\*
$SPAMMAIL


With this all mail with a Spam-Level of five or more will not be in your normal inbox but instead in a mailbox named spam in your mail folder.

As always, corrections and suggestions for improvement are welcome.

One thing puzzles me. Lots of people bought the Motorola Ming (or A1200 as it is also called). Lots of people are trying to make it work better, mostly with hacks (see the majority of the postings at MotorolaFans.Com). But there isn’t really a developer community that is writing software for it and addresses the issues that people are having.

Why? Simple. While Linux is open source and Motorola of course follows the rules of the GPL and makes the necessary sources available, they are not making available the necessary tools to build your own firmware image and flash it onto the phone. So even though I see that the calendar sucks (gee - you can’t tell from looking at it if a meeting starts on the hour or half past the hour? You’re kidding me!), I can’t simply replace it. Even though there are many small fixes that I’d like to apply (the battery meter is simply terrible; charge the phone for ten minutes and it shows fully charged, only to drop down to “red - about to die” after powering it on for 30 minutes - that’s useless), I can’t fix this.

I understand the need to control the radio stack and the access to the SIM and many issues concerning the network interaction. So put those in a signed binary and have the hardware refuse to operate when the signature of that binary can’t be verified. Tivo figured that out many years ago. But give us access to the smart part of this smartphone. I bet that if Motorola did that the community would come together and create the killer applications that would make more people want to buy this rather expensive phone. Which is clearly in Motorola’s best interest.

Unfortunately, it seems they don’t get it. Maybe this would be a way to address the issues that caused their stock price to drop like a rock last week…

I have used so many of them. The original Berkeley mail. Then elm, pine, vm (under Xemacs) and finally mutt. Those are all text console based and (at the risk of getting myself flamed here) are sorted in order of usefulness - with mutt clearly superior to the rest. They work exceptionally well if you don’t get a lot of HTML emails and if you don’t expect seamless integration of pictures, rich text documents and other attachments. Which, btw, until only a few years ago, meant they worked very well with the vast majority of email.

I also was exposed to the frightening class of gui-based email programs. The distressing Lotus Notes (which back then didn’t even speak the most basic Internet email protocols correctly - allegedly that’s fixed now). The utterly frightening Outlook Express. The omnipresent Outlook (which is not terrible as far as gui-based email programs go, but has all of their shortcomings that I’ll get to in a moment). Right now I use Entourage for work email - which in many ways is nicer than Outlook (for example, it runs on OS X and is reasonably well integrated into that which gives it a nice touch compared to Outlook), but in other ways worse (as it competes with Outlook, is from the same small software company in Redmond, WA, and still isn’t able to fully integrate with that same company’s Exchange server - how ridiculous is that? Entourage doesn’t understand MAPI and instead uses WebDAV to talk to Exchange - which simply takes a lot of potential features away).

And it’s sad to say, there’s a group of programs that’s even worse - the open source gui email programs (like Evolution, Thunderbird or Kmail). Why am I so negative? Well, they compete with Outlook and they don’t come even close. None of them can really integrate with the Exchange calendar (Evolution tries to but fails badly). None of them has a gui that’s even close to what Outlook or Entourage have to offer. They are slow (try using them with a 250MB mailbox under Exchange) and are simply hard to use - even allowing for the fact that gui-clients in general are bad for email…

Here, I said it again… so why do I dislike gui-clients so much when it comes to email? Simple. If you are dealing with a lot of email (and who isn’t, given the spam pandemic) then the number one task of an email client is to allow you to quickly sort, view and discard email based of a variety of criteria. Mail thread boring? Delete all emails in it. Mail author annoying you? Gone are his emails. Which other emails have I received from this person? Which emails where the subject contains the word “blog”?

Sure, you can do all of these with the gui programs. But that requires you to touch the mouse. Bzzzzt. Disqualified. If I get to an inbox with 400 new messages since yesterday evening (not unusual) I don’t have the time to keep moving from keyboard to mouse and back.

But let’s say for the sake of argument that there was a gui client that had a decent keyboard interface. That still leaves you with the problem that it will try to render all the stuff that people send you. Which is fine for the 5% of your email that you actually want to read in detail. And for the rest it is at best a waste (and with Outlook on Windows, often quite dangerous).

“But it’s so easy to use the gui clients!”, I hear you say. Yep, for the occasional or newby user. But once you spent some time with your email client (and again, this whole posting assumes that you get a serious amount of email - so you’ll be there soon enough) then all what makes the gui clients so easy to use at first now makes them even more annoying.

Yes, for people who love to send pictures or other embedded objects around to others, mutt is not as pretty. And the learning curve is steep. But I think it’s worth it. I use it every day for all my email at hohndel.org and just love it. Even though I read those email on Macs these days which means I’d have access to Mail.app - one of the better gui-clients out there. But a good text based mailer like mutt beats Mail.app for large volumes of email, any time.

So I decided to move from Blosxom to WordPress, first for my personal blog and then for my tech blog. And since I had about 550 postings and around 40 or so comments in my personal blog I needed a way to migrate my data. Googling didn’t find anything even remotely useful (the “import via RSS” suggestions simply lost too much formatting - things looked terrible, given how many pictures I have). Instead I figured I’d write a perl script that would do the hard work; pull all the postings and comments from Blosxom and import them into WordPress. Looking at the structure of the existing import scripts (and the fact that I know far less php than perl) I decided not to integrate this into WordPress but instead to insert the data directly into the mysql database. That should be fun. And amazingly it took not nearly as long as I feared!

Now I want to share what I learned with the rest of you, but the more I look at the script that I wrote, the more I realize that it is based on so many assumptions that it might be almost useless to anyone else. But then again, maybe it can help someone in a similar situation as a starting point. Writing it certainly helped me understand why WordPress doesn’t have an import function for Blosxom.

Here’s the fundamental idea of what I did

• install WordPress on the target system. One assumption made in the script is that you can access the mysql database from a system that has the Blosxom files accessible in its file system.
• set up the new blog. Depending on your needs you may have to find (or write) a theme that is similar to your Blosxom theme. In my case (the personal blog, not this one) the formatting of many of the postings was based on this being a fixed width theme of a certain width with certain classes defined in the CSS, certain margins set around different HTML objects, etc. So I started from something reasonably similar and then more or less wrote my own theme.
• delete the default posting and comment, make any other changes you want (blogroll, etc) and set up all your categories (important - the script will fail if it finds a category that doesn’t exist).
• back up your database
• I mean it. Use the wp-backup plugin. Or do it manually in mysql. But back it up. Really. I restored this backup quite a few times while working around bugs in the script, typos in the blog postings, etc.
• download the blosxomtowp.pl script.
• read the script. Edit the variables at the top of the script. Look through the assumptions made. Here are the ones I’m aware of, but you really might want to read through the script and compare with your file system layout, posting structure, etc.
  • it assumes that you have shell access to the machine that blosxom runs on and that you can connect to the WP mysql database from that machine
  • it assumes that you use directories under the main blosxom blog directory for your category hierarchy - just as with using the “categorytree” plugin
  • it assumes that you use the “meta” and “metadate” plugin to set the date on your postings (but it’s easy to change this to use the file time stamp instead - I just haven’t done that)
  • it assumes that you are using the “feedback” plugin for comments (but I think “writeback” and some others have similar file layouts and formats)
  • it assumes that you have already created /all/ categories that you have in blosxom in your WP database
  • it assumes the database table layout in WP-2.0.5

  Figure out what else you want to preserve (assuming you have different plugins than I had). Figure out what you can live without.

• you did back up the wp database, right?
• go to the main directory of your blosxom tree and run the script on one posting
  …/path/blosxomtowp.pl misc/aposting.blog (note that I used “.blog” as suffix - for most people that will be “.txt”).
• check your blog in a web browser. Did the posting show up? Does everything look right?
• start debugging. wp-phpmyadmin was a huge help for me to see what went wrong in the mysql database
• once this works for a few postings you can slurp all of it in (don’t forget to restore the backup, first, so you don’t get duplicate postings):
  find . -name \*.blog | xargs …/path/blosxomtowp.pl

I’m sure I’m forgetting a lot of things here. Please comment if you have additions, improvements, suggestions. The script is under the GPL, I’d be happy to accept fixes from anyone, but especially from people who actually are better at writing perl than I am (that’s not a high hurdle) and who can help me clean up the code.