flokrus Blog

(Dieser Artikel ist auch auf deutsch verfügbar.)

This article refers to a predecessor article concerning Time Machine backups on network shares in Leopard.

In the comments on this article a thread in Apple’s support forums was mentioned which describes problems with Time Machine and network shares (thanks to John M for that remark).

To explain the problem you need to be aware of how sparse bundle images (the ones Time Machine uses for network backups) work. Sparse bundle images and sparse images are special forms of disk images with certain advantages. To say it in simple terms a disk image is just a file that is treated like a drive (e. g. a hard disk). The image file is contained in a real drive (locally or on the network) which I will call container.

You can mount the image and use it like a real drive. Mac OS X takes care of writing the data into the image file. In the simplest case the image is a file that has exactly the size of the virtual drive’s maximum capacity. If you create an image with 100 GB capacity you will have a 100 GB file even if the image only contains some MB of data.

And here the sparse images come into play. They occupy only the space the data in the image occupies. Therefore, you can create an image with 100 GB capacity which as a file is only a few MB big. The image file grows as it is filled with data.

However, there is a catch in it. Deleting files in the image does not result in a smaller image file. For example: If you create a 10 GB file in the image the image file will increase to a size of 10 GB as well. Free space in the image will be 90 GB. Afterwards you delete the file and the free space in the image will be 100 GB again. Without further intervention the image file however will keep its size of 10 GB*. Inside the image everything is in order. Thus, while using the image its file will at most keep its size, mostly grow in size (due to creating further files in the image), but never shrink by itself. This is one part of the problem.

The other part of the problem is a behaviour of Mac OS X which is actually really useful. Mac OS X does not prevent the user from creating a sparse image with a maximum capacity bigger than the free space or even the maximum capacity of the container. This may turn out to be a problem. For example take our fresh image with 100 GB capacity. Assume its in a container with 20 GB free space. The user might want to create a 30 GB file in the image which naturally will not work.

Mac OS X acts rather clever here. If an image is mounted Mac OS X checks how much free space is available in the container (here 20 GB). If this value is smaller than the theoretically free space in the image (here approx. 100 GB) Mac OS X will limit the free space in the image to the free space of the container. In our example: The image has a maximum capacity of 100 GB and 20 GB free space even if there are no files on the image!

In itself this behaviour makes sense. In combination with Time Machine it can lead to fatal consequences. For example: In our previously mentioned image you create an 18 GB file. The image file will grow to a size of 18 GB. Afterwards free space in the container and the image will be 2 GB. Now you delete the 18 GB file. Theoretically the free space in the image will increase by 18 GB.

Now both above mentioned effects come into play. The image file will keep its size of approx. 18 GB. Hence the free space in the container will still be 2 GB. Therefore, the free size in the image will as well be only 2 GB. In this case deleting a file in the image does not create further free space in it. If you want to create a new 18 GB file Mac OS X will not allow that. The problem can be avoided by always keeping enough free space in the container. In our case we would have to free at least 16 GB in the container*.

Assume in our initial situation above free space in the container would not be 20 GB but 120 GB. Mac OS X will then always specify the free space regarding only the image (100 GB capacity). Deleting the 18 GB file will indeed increase the free space to 100 GB. This also shows how fatal it is, if the maximum capacity (or the maximum free space respectively) of the container is less than the capacity of the image. In this case the free space of the image will always be specified regarding the container. At some point the image will become unusable because the image file grows inexorably while the apparent free space of the image shrinks.

This is exactly the trap Time Machine falls into, if you do not pay attention to the free space on the network drive. Assume the Time Machine image has less free space than the next backup needs. Time Machine will then automatically delete old backups to free up some additional space. If the free space on the network share is greater than that in the image (i. e. the free space in the image is not affected by the free space in the container) everything will work smoothly. With every deleted old backup the free size in the image will grow a little.

However, if the free size in the image is limited by the container deleting backups will not create additional free space. Time Machine will futilely keep on deleting all backups but the last one in order to clear space. Afterwards it will fail due to lack of free space (like in the above mentioned support thread).

Possible trouble-shootings:

1. Make sure there is enough free space on the network drive in the first place and keep it free permanently (i. e. at least the maximum size of the image). The image capacity must not exceed the free space of the container.
2. Regularly free up additional space on the network share so that there is always enough free space for the next backup. In the end this leads to option 1 and needs more attention. However, while the backup image is still small you can use the free space for other things. The image capacity must not exceed the maximum free space of the container.
3. Regularly delete old backups in Time Machine manually and reclaim freed space via hdiutil. Most complicated but also the most flexible solution.

Disclaimer: I have never experienced these problems in Time Machine myself. Therefore, I cannot guarantee that the above stated solutions actually work. In my case there is enough free space on the network share so that backups are created smoothly. Your own reports and experiences are very appreciated.

You should always be aware that Time Machine backups on network shares are not officially supported by Mac OS X (except Time Capsule). In doubt you should not trust a solution like this.

* There is a possibility to reclaim unoccupied space via hdiutil. However, as Time Machine does not utilize it I will ignore this possibility here.

This is a slightly shortened version of an article I already wrote in German which is my native language so please forgive me possible mistakes.

Mac OS X features in its newest release Leopard (Mac OS X 10.5.2 while I am writing this) an automatic backup solution called Time Machine. It automatically creates backups, takes care of everything and even incorporates a very stylish interface to find and recover old files.

Unfortunately Time Machine has some limitations. So far it only works with external hard disks that are connected directly to the Mac. Apple now provides a solution for network backups called Time Capsule. However, using already existing drives for backup would be desirable.

You can find guides to make Time Machine at least accept network shares as backup drives in the web. Using the command (all in one line)

defaults write com.apple.systempreferences
                      TMShowUnsupportedNetworkVolumes 1

in Terminal is sufficient. But this only results in accepting the share. At least in my case creating the first backup failed with the error:

Time Machine Error - The backup disk image could not be created.

Pretty dissatisfactory. We gonna need new solutions. In my investigation I often found the statement that Time Machine simply would accept self made disk images on the network share. It doesn’t.

It rather helps to investigate the failed backup attempt more precisely. You will notice that Time Machine tries to create a sparse bundle disk image on the share. It remains Time Machine’s mystery why that fails.

On Apple’s support forums you’ll find further advice. It is proposed to copy the bundle image while creation and to copy it back after this fails. Did not work for me. Anyway, the solution is much simpler. During the backup attempt Time Machine creates the bundle image on the share. This is called <computer’s name>_<string>.sparsebundle (Update: by the way, string is the MAC address of the local en0 device without colons). You should keep the exact name in mind (e.g. by copy & paste or by looking in Console or by constructing it yourself from computer’s name and MAC address).

Afterwards you start Disk Utility and create another “sparse bundle disk image” on your local drive. Volume Size should be the designated maximum size (see update 2) of the Time Machine backup (don’t be afraid, only by using the image it will increase in size). Further parameters: Volume Format - Mac OS Extended (Journaled); Partitions - No partition map.

After this use the exact name Time Machine assigned to its bundle image to rename your newly created image (if not already done while creating your own image). Finally you copy the image on the share and manually start a backup via menu bar.

Voilà. Afterwards Time Machine should start the tedious initial backup.

Update: Giving it a try I am positively surprised. After the successful initial backup Time Machine tries to create new backups every hour. And doing so it behaves very ideally. If the network share is not accessible (e.g. host is turned off) the backup simply will not be performed (and the user won’t be bugged with error messages). Nevertheless Time Machine continues to try to access it hourly. If the host comes online again new backups will be created. The needed share and disk image will be mounted automatically.

If you want to access Time Machine to recover files the automatic mounting will work as well. At first (in my case) the Spotlight index was broken. I don’t know why this happened, but I have created an article how to repair this index (currently only available in German).

On the network host which provides the share I have created a special backup user whose home directory contains the image. Fortunately, Time Machine has no problem if a user connects to shares on the host with his own user data.

Let’s face the drawbacks. Once a backup is created or Time Machine is launched the image will indeed be mounted but will not always be unmounted afterwards. Thus, users will have the image on their desktops. The connection to the share itself will also be kept established (but it is only visible by entering mount in Terminal).

Both isn’t necessarily bad if the share keeps being available. However, this does not have to be the case. If you want to disconnect image and share correctly from your Mac you’ll first have to unmount the image itself. Sometimes the system requires an admin password to do so. The share can only be unmounted by calling umount in Terminal. Supposably it will become problematic if the host is shut down while a connection is established or even a backup is in progress.

And of course bandwidth is a major drawback. If (as in my case) the Mac is a MacBook primarily using 802.11g WLAN and accessing the share on a Windows machine via SMB you surely should not expect too much from this solution.

Update 2: Things you should know about the maximum size of the image and the free space on the network share.