Synology DSM and Veeam 11

For a long time, Veeam has been telling its users to not use "low-end NAS boxes" (eg, Synology, QNAP, Thecus) as backup repositories for Backup & Replication (VBR), even though these Linux-based devices should be compatible if they have "x86" architecture (as opposed to ARM).

The reality is that none of these devices use "bog standard" Linux distributions, and due to their appliance-based nature, have some significant limitations on what can be done to their custom distributions.

However, there are many folks—both as home users or within small/budget-limited businesses—who are willing to "take their lumps" and give these things a shot as repositories.

I am one of them, particularly for my home "lab" environment. I've written about this use case (in particular, the headaches) a couple of times in this blog [1, 2], and this post joins them, addressing yet another fix/workaround that I've had to implement.

Background

I use a couple of different Synology boxes for backup purposes, but the one I'm dealing with today is the DS1817+. It has a 10GbE interface for connectivity to my network, a quad-core processor (the Intel Atom C2538) and 8GB RAM (upgradable to 16GB, but I haven't seen the demand that would require it). It is populated with 8x1TB SATA SSDs for ~6TB of backup capacity.

I upgraded DSM to 7.0 a while back, and had to make some adjustments to the NFS target service to continue to support ESXi datastores via NFS 4.1

Yesterday, I updated it to 7.1-42661 Update 2, and was greeted to a number of failed backup jobs this morning.

Symptoms

All the failed jobs have uniform symptoms: Timeout to start agent

With further investigation, I saw that my DS1817+ managed server was "not available", and when attempting to get VBR to re-establish control, kept getting the same error with the installation of transport services:

Installing Veeam Data Mover service Error: Failed to invoke command /opt/veeam/transport/veeamtransport --install 6162:  /opt/veeam/transport/veeamtransport: error while loading shared libraries: libacl.so.1: cannot open shared object file: No such file or directory

Failed to invoke command /opt/veeam/transport/veeamtransport --install 6162:  opt/veeam/transport/veeamtransport: error while loading shared libraries: libacl.so.1: cannot open shared object file: No such file or directory

Workaround

After failing to find a fix after some Linux-related searches, I discovered a thread on the Veeam Community Forum that addressed this exact issue [3]. 

This is apparently a known issue with VBR11 and Synology boxes, and as Veeam is moving further and further away from the "on the fly" deployment of the transport agent to a permanently-installed "Data Mover" daemon (which is necessary to provide the Immutable Backup feature), it becomes a bigger issue. Veeam has no control over the distribution—and would just as soon have clients use other architectures—and Synology would probably be happy with customers considering their own backup tool over competing options...

At any rate, some smart people posted workarounds to the issue after doing their own research, and I'm re-posting for my own reference because it worked for me.

1. Download the latest ACL library from Debian source mirrors. The one I used—and the one in the Forum thread—is http://ftp.debian.org/debian/pool/main/a/acl/libacl1_2.2.53-10_amd64.deb
2. Unpack the .deb file using 7zip
3. Upload the data.tar file to your Synology box. Feel free to rename the file to retain your sanity; I did.
4. Extract the tarball to the root directory using the "-C /" argument:
   tar xvf data.tar -C /
5. If you are using a non-root account to do this work, you'll need to use "sudo" to write to the root. You will also need to adjust owner/permissions on the extracted directories & files:
   sudo tar xvf data.tar -C /
   sudo chown -R root:root /usr/lib/x86_64-linux-gnu
   sudo chmod -R 755 /usr/lib/x86_64-linux-gnu
6. Create soft links for these files in the boxes filesystem:
   sudo ln -sf /usr/lib/x86_64-linux-gnu/libacl.so.1 /usr/lib/libacl.so.1
   sudo ln -sf /usr/lib/x86_64-linux-gnu/libacl.so.1.1.2253 /usr/lib/libacl.so.1.1.2253
7. Last, get rid of any previous "debris" from failed transport installations
   sudo rm -R /opt/veeam

Once the Synology is prepped, you must go back into VBR and re-synchronize with the Linux repository:

1. Select the "Backup Infrastructure" node in the VBR console
2. Select the Linux node under Managed Servers
3. Right-click on the Synology box being updated and select "Properties..." from the popup menu.
4. Click [Next >] until the only option is [Finish]. On the way, you should see that the Synology is correctly identified as a compatible Linux box, and the new Data Mover transport service is successfully installed.

Summary

I can't guarantee that this will work after a future update of DSM, and there may come a day when other libraries are "broken" by updates to VBR or DSM. But this workaround was successful for me.

Update

The workaround has persisted through a set of updates to DSM7. I have seen this come up with DSM6, but this workaround does not work on that; too many platform incompatibilities, I suspect. Need to do some more research & experimentation for DSM6...