Community Matters

Excellent post by my friend Larry Augustin on the need to stop using the patent encumbered FAT filesystem.

I usually don’t just post links to other posts here, but I think this one is important. We do indeed act as if FAT wasn’t patent encumbered. And this touches both of my biggest hobbies – Linux and photography.

We as an industry (and collectively, as consumers) need to stop relying on this rather mediocre filesystem. There are much better choices available. Larry suggests ext2. That’s under GPL – maybe that’s an issue for some people. But there are really good BSD licensed filesystems out there. Without having done much research on the issue, maybe UFS would be a better choice? A lot of common operating systems support their own version of it already – so this might be a great starting point.

The industry would need to get together and standardize one standard version of it, but since everyone would have to give up something of their proprietary flavor, maybe that wouldn’t be so hard (and again, it would certainly be easier than dealing with a GPL licensed filesystem).

Update: Arjan pointed out that I was too quick in discounting Larry’s suggestion. There are in fact a number of BSD/MIT license style implementations of ext2 – there’s even a version for Windows and the Mac. So maybe ext2 is the best starting point after all… doing some searching around it seems that few of these projects are actively maintained, but that should be something the open source community could easily tackle…

…and other systems with cheap and slow SSDs like the Dell Inspiron Mini 9 or Acer Aspire One.

Important note: this is not written for ultrafast SSDs like the Intel SSDs found in high end laptops and servers; it’s targeting the type of SSDs usually found in netbooks. And of course it also doesn’t apply to hard disk based netbooks.

The problem? These very cheap SSDs tend to have terrible write performance. And filesystem designers apparently never considered the impact of devices that have fabulous access times, decent read speed but that come to a screeching halt when sending them too many writes at a time.

Most Linux distributions by default install ext3. Whether or not a journaling filesystem is a good idea for an SSD appears to be still up to debate; but unless the SSD does decent wear-leveling I would stay away from simple journals like what ext3 provides which keep overwriting the same part of the ‘disk’ over and over again.

Also, by default Linux tracks modifications to the access time of every inode. So every read from disk also creates a write updating the atime in the inode. To make things work better on a cheap SSD it’s best to turn this off using the mount option noatime.

The third problem you keep running into is the frequent fsync issue that I’ve described earlier. Here your best bet is a very current Linux distribution that has the latest versions of some of the applications that use fsync more heavily. This is certainly an area where Moblin is trying to be careful.

The fourth problem is the apparently common assumption that a Linux system has to have a swap partition. As it turns out, swap partitions are actually rather counter productive in a system as discussed here. As your system gets more heavily loaded (more things are running and competing for the limited write bandwidth to the SSD) you are now adding the Linux kernel as another creator of write accesses to the SSD – as it is trying to swap out pages to free memory. It’s much better to rely on Linux paging algorithm to keep fewer read only pages in memory – these are read back from “disk” when they are needed again, but as they never get modified they don’t need to be written out to the SSD. Yes, this means that you may run out of memory sooner – but if you do, that may be a sign that you are trying to do things that the little netbook wasn’t designed for (don’t run VirtualBox on it – be careful how many office and email applications you have open at the same time – and yes, limit the number of tabs in Firefox and restart it every once in a while.. Firefox is a real memory hog).

So what do I recommend? Long term it’s BTRFS. I am using this for some tests on one of my EeePCs and the difference is stunning. But BTRFS is far from being stable enough for real life and isn’t (and won’t be for some time) used in any Linux distributions.

In the meantime I’d go with ext2 with noatime and no swap partition. If your system (like most EeePCs) has two separate SSDs with different write speed, make sure that directories that are frequently written to (or fsync’ed) are located on that drive. I have explained how to do this in my posting on solving the Firefox and Evolution fsync issue.

Oh, I’ve played with JFS, XFS and yes, even vfat as possible alternatives. Neither of them provides any noticeable advantage over ext2 – and since they are less frequently used I am concerned about potentially running into bugs and problems that haven’t been discovered by other users, yet.

I posted a while ago about Nikon’s lack of large aperture prime lenses. And apparently this problem hasn’t escaped Nikon’s attention. After releasing an updated AF-S 50mm f/1.4G a few weeks ago, Nikon has added yet another large aperture prime, this time it’s first large apperture lens that is designed for DX cameras, the AF-S DX 35mm f/1.8G (in case you wonder, their first DX prime lens was the AF DX 10.5mm f/2.8G fisheye).

This lens doesn’t replace the full frame AF 35mm f/2.0D but instead adds a cheaper, smaller, lighter and faster lens especially for DX cameras. Reactions online have been a bit mixed, some people lamenting that it’s “only” a 1.8 and “only” a DX lens, but I think these people are missing the boat. Most Nikon shooters these days have a DX camera. And unless they are seriously considering a switch to one of the FX bodies any time soon, for them an FX prime just adds unnecessary weight and size and cost. Yes, I’d rather have seen an f/1.4 lens and I’d have preferred a 30mm or 28mm lens as their first entry (I tend to find a “52mm-equivalent” lens a little too long for many typical low-light situations, e.g. shooting indoors), but I think this is a great step in the right direction and hopefully we’ll see several more lenses follow, maybe even the new 28mm f/1.4 lens that so many of us are waiting for.

I certainly am going to look at buying the lens once it comes out (so far no one appears to take pre-orders, yet).

This seems to be a feature that doesn’t get a lot of attention – but I think it’s widely underestimated. Nikon DSLRs like the D300, D700, D3, D3x, D90 (and many of the older ones – but I don’t know exactly when this started) allow you to use the ISO sensitivity like a third parameter when calculating exposure!

Traditionally people think of the ISO value as fixed (I guess this comes from the day of using film that has a given ISO value) and then look for a matching f-stop and shutter speed that creates correct exposure.

The f-stop is limited by what your lens has to offer. f/1.4 or 1.8 on a fast prime lens like the 50mm/f1.4. f/2.8 on an expensive zoom like the Nikon 24-70mm/f2.8 I played with last weekend. Or something slower like f/5.6 on the cheap “kit lenses” most cameras get sold with.

And the shutter speed is limited by the slowest speed that you can take a picture at without it getting fuzzy because of the jitter of your hands. In the old days people used the rule of thumb of 1 divided by the focal length of your lens (so 1/50s for a 50mm lens, etc). Today with VR you usually can push this quite a bit slower – I often take shots at 1/50s with a 200mm lens.

But what do you do if you are in a low light situation? Indoors without flash, in the shade, at dawn, etc? Well you could of course increase your ISO value, but likely you’ll get it wrong. And if you crank up your ISO too high you get too much noise, but if you don’t go high enough, your shutter speed will be to slow and you will get a picture that isn’t crisp. Oh and this of course takes time and attention and you risk losing that perfect shot you’ve been waiting for.

The solution: auto-ISO. Here you can tell your Nikon the slowest shutter speed you are willing to allow. And you set your ISO to a reasonable value that gets you low-noise pictures (I tend to have it at ISO200). Then as long as there is enough light, you’ll get the best possible picture at ISO200. But once it gets darker, instead of forcing a slow shutter speed, the camera increases the ISO in order to keep the shutter speed no slower than the value that you picked. I much rather take a picture at maximum aperture, 1/50s and ISO800 than at 1/12s and ISO200. The first one will be nice and crisp, the second one is almost certain to be fuzzy due to the jitter of my hands when taking the picture – and potentially the movement of my subject.

Is this a miracle cure? Of course not. At some point it’s just too dark. Depending on the camera you use, there’s an upper limit to useful ISO values. A D300 takes great pictures up to about ISO1000 (some noise, but easy to deal with). A D3 or D700 allows you to go all the way to ISO3200 and still be happy with most pictures. But once you go beyond that, you’ll still get noisy pictures.

But this still adds something like 3-4 more f-stops to what you can do with your camera without this (and without having to manually change the ISO setting – which likely means you’ll miss the shot).

Try it out!