Thoughts on Filesystem Granularity

One approach to disk partitioning (and certainly the easiest) is to have as few partitions as possible, e.g. I've seen some installations with only two partitions: one to hold the operating system, and one swap partition. Not only do I feel this is to be unwise, but I tend to proselytize for the other extreme: many disk partitions, high filesystem granularity.

The reason for this is that over the years I've seen situations where some daemon went bonkers, logging voluminous amounts of verbiage to something in /var/log, thus filling up /var (or worse, if /var wasn't a separate partition of its own, filling up the root filesystem); or where some user copied (or tried to copy) a massive file to /var/tmp, with the same result; and so on.

If /var, /var/log, and /var/tmp are kept in separate disk partitions, it helps quarantine the damage when some daemon (or some user) gobbles up all the available free space in one of them. If a user fills up /var/tmp, at least it doesn't keep syslogd from continuing to log information in /var/log; if some daemon generates so much syslog verbiage that /var/log fills up, at least other things which need to create (or append to) files elsewhere in /var are still able to do so; and so on.

So the filesystem layout I prefer has at least this level of granularity:

Linux systems should also have a separate partition, and systems using {PBS|torque} (or anything else which is likely to create a lot of output in /var/spool) should probably also have a separate
partition, since PBS has occasionally gone into “spew” mode, dumping lots of stuff into /var/spool. If adequate disk space exists for other (i.e. non-OS) filesystem partitions, they should be mounted on subdirectories of /local, e.g.

It's worth noting that an advantage of having /usr/local in a separate partition is that if it's ever necessary to blow away the OS, scrub the partitions, and reinstall from scratch, it's easy to do so while still keeping /usr/local untouched. And in the case of re-imaging a cluster node with xCAT or SystemImager, the OS can be installed without blowing away the previous contents of /var/log, which is often useful to keep around.