Community Matters

So this is NOT the mail program I have written about before. It is (for now, at least), squarely targeting geeks. You want to know how geeky? The current user interface for Notmuch is Emacs. Yes, Emacs.

Ok, if you stopped laughing, I’ll tell you why I am still quite excited. Notmuch is entirely search based. It tries to do one thing and do it well. Manage the tons of email that many of us get. Have you ever tried to manage several GBs of email, hundreds of thousands of messages with one of today’s “standard” email programs like Outlook, Evolution, Thunderbird or Mail.app? Don’t – unless you are a very patient person.

After a rather lengthy initial import of your email (this can take a few hours, depending how fast your disk is and just how many gigabytes of email you have), most every operation is really snappy. Think someone sent you an email about kittens to your home address? Something like notmuch search tag:tome and subject:kittens will find this email in no time at all. And you can get arbitrarily sophisticated here – after all, these are all just tags. notmuch search tag:unread and tag:lkml and from:torvalds@linuxfoundation shows you the emails that Linus has sent to LKML that you haven’t read (assuming that you’ve set up an lkml tag in your filtering of incoming emails). The possibilities are endless.

So set up some rules on your incoming email. Classify depending on sender, subject, mailing list, topic, almost anything you can think of. And you can search for any combination of these tags – and of course a full text search of the mail bodies. Instead of looking at mail folders you now simply looked at saved searches – you can give names to them and treat them just like folders – except of course they are virtual and near-instantaneous.

Sounds complicated? Try “incredibly powerful”. More so than Gmail. And on your local machine, not somewhere in the cloud with someone else in control of your data.

Notmuch is at version 0,3.1 as of this writing. And there’s still a lot of work that needs to go into it (and especially into a more non-geek friendly UI) – but what is there already is so powerful that I have switched to notmuch as my mail application – and have dusted off my emacs foo to use it within emacs. And even started hacking elisp (and the backend, written in C and C++) to contribute.

Give it a try. You might be surprised.

I like the idea of little “servers” (and I use that term lightly) that are quiet and cool. I have replaced one of the Mac Minis that I used as a server at home with a fanless little system based on an Intel Atom Z530 CPU (a Portwell WEBS2120). 2Gigs of memory, a small SSD and you have a completely quiet and reasonably powerful system. Definitely works as a little web / mail / openvpn / DNS server for your home network. Cheap, cheap to run, flexible.

I expect we’ll see a lot more of these kind of systems used by all the web hosters. These are great little dedicated servers – and since they barely get warm they should keep operating costs low.

It seems that many of the low cost hosting companies have started to offer Atom based dedicated servers at VPS prices. So far these seem to be Atom N270/280 or even new N450 based system. But I’m sure the Atom Z5xx based ones can’t be far behind.

This has been an ongoing frustration for me for a long time.

I used to be a mutt user – still one of the best email clients out there. That is, if your email comes from an IMAP server and is mostly text-only. No images, HTML, links, etc. Yes, it can sort of kinda work with those things, but please, let’s get real.

Emacs and its various mail modes are of course an interesting option (especially as you can, in fact, display most everything inside modern emacs). And Notmuch is making handling tons of email even easier with decent emacs integration (yes, it’s very early in its development, but for things like reading lkml it is amazing).

Or you can go with Thunderbird (version 3 is really impressive, the tabbed UI takes a little getting used to but then worked rather well for me). Or claws-mail (fast but rather unstable and its single-threadedness really got me to hate it). Or even (yikes) Evolution. Sadly without a strong leader anymore and rather aimless for the last year or so.

But the problem is this – if you want to access work email as well as your personal stuff, chances are that you are forced to integrate with MS Exchange. I can give you tons of reasons why Exchange is a Really Bad Idea™, but of course your corporate IT department is likely to ignore those and tell you “Exchange it is”. And among the biggest flaws of Exchange is its rotten IMAP support. Incredibly slow, barely standard compliant (actually, there are a bunch of annoying bugs). And if you want calendar integration (arguably the best feature in Exchange) there is no good way around MAPI (at least not with Exchange 2007).

And that’s where open source email clients really fail. The only one with even attempted MAPI integration is Evolution. And that is one of the weakest parts of Evolution. Extremely unstable, slow, and so frequently flat out broken that I cannot really suggest using it for day to day work. Emails disappear, or their envelope is there and no content, parts of the headers are missing. The calendar is completely hit or miss: the latest version seems to get my single-instances meetings correct if they come from another user, get the time zone wrong if I enter them myself on the Blackberry or via OWA, and seems to completely miss out recurring meetings that were NOT entered by me. Not useful if I need to be able to rely on my calendar being correct (which is, after all, the point of a calendar).

So… what I do today is offlineimap to get emails from Exchange (or any other IMAP server) into a set of local MailDirs (this hides the latency of the IMAP implementation – especially important with Exchange), then Evolution to read that email locally and OWA for calendar.

Really, not a good solution at all. We need a decent MAPI client. The libraries are all there, the communication with the Exchange server is relatively easy to set up. What’s missing is an acceptable front end that can deal with the typical mess of email that people get (Thunderbird seems to be a good start and appears reasonably active and well maintained), that can do calendaring (again, Thunderbird with Lightening could do the job) and that has a reasonable UI, good keyboard shortcuts for the power users and most importantly is fast. So I guess we need MAPI integration into Thunderbird. Any takers?

I’ve been fighting to get TweetDeck to work on my Linux system for a while. There simply is no comparable native client under Linux. I’ve used Gwibber which is ok, but no comparison to TweetDeck.

There are a couple of problems to solve: first, you need to get Adobe Air to work. And with all due respect to Adobe – they clearly haven’t figured out the kinks to making their software actually install easily on the various Linux distributions. A quick Google search seems to make that painfully clear.

Forget 64bit Linux. Yes, allegedly it works for a few people with various 32bit libraries installed, but after six weeks of trying to get this to work I came to the conclusion that this was a lost cause.

And even with 32bit Fedora-11 there still are a number of problems to solve. First you need to make sure that you have all the dependencies installed – even though it would be easy to have rpm do that for you, Adobe clearly hasn’t figured out how to do that… so you have to do this manually:

sudo yum -y install gnome-keyring rpm-build nss

Then (thanks to erik jacobs) you appear to need to manually create another link for librpmbuild:

sudo ln -s librpmbuild.so.0.0.0 /usr/lib/librpmbuild-4.7.so

Now you are ready to run the installer:

chmod +x AdobeAIRInstaller.bin
sudo AdobeAIRInstaller.bin

But this still doesn’t solve the problem of installing AIR applications. Adobe wants to install them into /opt by default (which a regular user can’t write to) – and even after changing that to do writeable by my user things still failed with cryptic (and useless) error messages. So I finally figured out that I needed to manually download the AIR installer packages (like TweetDeck_x_yz.air) and then run the AIR application installer from hand (again as root):

sudo Adobe\ AIR\ Application\ Installer

and then pick the .air file in the file select box; the installer is too dumb to allow you to pass a .air file on the command line. Come on guys…

With all these steps I got it to work – but frankly I think this is an embarrassing sign for how much further AIR has to go to be really useful on Linux. 2 out of 10 points, Adobe…

After seeing hosting providers offer Atom based dedicated servers I started wondering how they are doing that? A pile of EeePCs? Unlikely.

Turns out there are a couple of companies offering blade servers with Atom blades – one is SERVER8 in Italy which seems to have a very smart approach – off the shelf Micro-ATX or Mini-ITX motherboards, including Atom-based motherboards can be used in a custom 6U case following the Open Blade spec.

SuperMicro offers 1U Atom-based servers and has announced an Atom-based blade server as well. There are do it yourself 1U rack mounted servers available. And I’m sure there are more similar offerings out there.

In general I guess server processors are the better fit for servers – but for this interesting niche market of people who want a dedicated server but don’t need a ton of performance (i.e. for things like hosting your own blog) these atom-servers could really be the perfect solution.

After posting about hosting your own blog a couple of days ago, one of my co-workers pointed out to me an interesting new trend that he has seen… more and more server hosting providers offer entry level servers based on Intel Atom motherboards at prices that start to compete with virtual private servers.

That’s an interesting use of the Atom processor (which I tend to think of as the netbook CPU). But for many typical blog hosting scenarios a system like this offers plenty of performance with all the advantages of having your own dedicated server – and really good prices.

InterServer offers a dedicated server (cutely named “VPS Buster”) for $39/mo. Others offer similar pricing. Nice.

(and no, I don’t get money from linking to them, sadly. It was just the cheapest that I found in a few minutes of googling – feel free to comment if you see better deals elsewhere)

I have never hosted any of my personal blogs anywhere else. And every time I talk to people who do I am happy that I was never tempted. There are way too many issues with doing that. Lack of control would be my number one concern. I want to be able to decide which OS I’m running, which version of the web server, which libraries I have available. Which blogging software and which version of it. Etc.

If you host with Blogger or Typepad or even on WordPress.com you are restricted to the versions someone else is willing to give you. You can’t change the underlying blogging tool, can’t install a new library – often can’t even install a plugin.

The other day I started hosting my wife’s blog. That brought it home to me. You want a development blog? Sure, no problem. Let me add another WordPress instance under a different hostname. You need the GD library? No problem, apt-get install php5-gd and that’s taken care of (I decided to run Debian on my servers quite a while ago). There’s a problem with xyz? Let me take a look in the log file. Very powerful. Very liberating.

Yes, some of the hosters like Dreamhost allow you something almost as good. With lots of choices and lots of control. But still, you’re in a jail – it’s just bigger and more flexible. The only way to really control what you are doing is to host the blog yourself. On a VPS or (like this blog) on a dedicated server.

After sitting on the sidelines for a long time I finally figured “what the heck” and started to use Twitter.

You can find me as dhohndel.

Over the past couple of months this blog has moved twice. I think it was fairly seamless and I also think that the difference in responsiveness of the blog is fairly dramatic.

The first move was from my server at home to a VPS hosted with Datarealm. The rationale was that I finally wanted the webserver in a real data center – the data volume was reaching a point where it became noticeable for me when competing for bandwidth with it.

And then the second move this week from that VPS to a small dedicated server from Rackmounted – which actually is the same company! This one is a bit harder to explain. The VPS was normally fast enough to serve the blog (there was another VPS on the same server running amok for a few days that impacted performance, though). But the VPS has limited bandwidth and under load created a user experience on the blog that I found simply disappointing. So I figured I’d try a dedicated server.

I searched hard to find a hosting service that is at the same time cheap (affordable?) and reliable and provides good customer service. I searched the forums, I contacted some of the budget vendors. Scary stuff – response times of three days and more, sales people and technical support people who clearly know very little about the infrastructure and systems that they sell. And in general a lack of “service attitude” that was disheartening. Worstcase example was WSServer (I won’t provide a link here – they don’t deserve that). But several others weren’t much better.

Rackmounted on the other hand was a positive surprise. It’s a small company of about nine people that cover both the VPS and the dedicated server side of the the house. They respond to requests within hours (and sometimes minutes). They try to understand what you are trying to do. They don’t assume that you are stupid (this sounds silly, but let me tell you, it makes all the difference if the tech support person you talk to assumes that you know what you are doing and tries to work with you instead of trying to get rid of you).

The coolest thing was when I told them that I was looking at a dedicated server but wasn’t really able to justify the expense of the servers that they were offering. I asked if they had any specials, any old hardware that they could offer at a discount (and flat out told them the price range that I was looking for). And they got right back to me, made me an offer that I couldn’t refuse and set it up right away.

Really cool. Really surprising. And I’m happy to recommend them to anyone looking for affordable high quality hosting. Regardless of VPS or dedicated server.

The performance difference to the VPS is stunning. I’m very happy.

During the panel that I participated in at OSBC the panelists were asked what they thought of the “Google model” of computing. I was rather negative about this. Actually, according to Yahoo News, I said “The Google model really scares me”. Yep, that sounds about right. I talked about this before. Where’s my data and Google’s subpoena is a sign of some of the downside of their server centric approach… . So consider this part three of an ongoing series…

Here’s a new example why the cloud model is flawed – and just for kicks, it’s not picking on Google but on Adobe this time. Adobe recently released their Web version of Photoshop Express. If you ignore the cute “the lawyers make us do this” tool-tips and actually click through the terms of use you’ll find language like this:

Adobe does not claim ownership of Your Content. However, with respect to Your Content that you submit or make available for inclusion on publicly accessible areas of the Services, you grant Adobe a worldwide, royalty-free, nonexclusive, perpetual, irrevocable, and fully sublicensable license to use, distribute, derive revenue or other remuneration from, reproduce, modify, adapt, publish, translate, publicly perform and publicly display such Content (in whole or in part) and to incorporate such Content into other Materials or works in any format or medium now known or later developed.

So when you use their web software you grant Adobe worldwide, royalty-free, perpetual, irrevocable and sublicensable rights to use, reproduce, modify(!), publish(!) and derive revenue(!) from your images.

Wow. One more reason why I don’t like these “cloud services”.