fighting the lack of good ideas

don’t use symlinks unless you *know* you can

I first ran into this on Solaris in the context of [then] Opsware SAS (then HP SA, now owned by Microfocus). Bind mounts might be OK … so unless the tarball has symlinks included, don’t use them – they get traversed differently than “real” directories.

In short, when directory traversals are done, sometimes it looks at the permissions bits and if the first character is not a d (for a symlink, it’s always an l), many processes can fail.

Symlinking files is [possibly] a different story: though permissions are usually wonky on symlinks (most often lrwxrwxrwx vs -rw-r--r--, for example), since you cannot traverse into a file (whereas you can into a directory), it’s generally ok

Also – sometimes when directory listings are pulled, the symlink is fully-dereferenced, and something that appears to be in, say, $SPLUNK_HOME/etc/deployment_apps but is really in, say, /some/other/place, there are some times when Splunk will decide not to deploy it, because it’s not where it “belongs”.

Also – checksums can be computed on the symlink and not the actual file, in some (perhaps all) instances: so if, for example, you have the same outputs.conf in several apps by way of symlink, and you change it in one, the checksum for all the others may (and typically do) not get updated … so you can be left in an inconsistent state for your configs (because not all locations that should’ve received the updated outputs.conf have received it, since they’re symlinks and not a real file, and the checksum may not update on those particular apps).

Moral of the story?

Unless you really know what you’re doing, never use symlinks with Splunk.

symlinks and nfs

I recently discovered an interesting “feature” of symbolic links in conjunction with NFS mounts: they don’t work!

For example, let’s say you have the following NFS export:


Inside of that export, you have the following path:


In *this* directory, you have a symlink called ‘current‘, which points to a different location:

current -> /media/osmedia/linux/ubuntu/11.10

Here’s the rub: if anyone tries to mount the root (or sub path) of that NFS export (say /media/files/isos/osmedia) at a local point (say, /mnt/osmedia), the symlink will fail to work correctly because there is no path /media/osmedia/linux/ubuntu/11.10 on the local machine!

That’s not cool.

Here is another case where bind mounts will save your bacon!

Instead of using symlinks on the NFS host, use bind mounts – that way the path will still work when exported.