Recently a customer reported an issue with a backup copy job. At this customer we implemented a new leased line with synchronous 1Gbit/s bandwidth from the main site to a secondary site. That gave us completely new possibilities regarding the backup of data and making it available. There is no need for a WAN Accelerator anymore, we can replicate VM’s with Veeam directly to the secondary site. And also backup copy jobs to this site go now A) directly and B) in a regular manner (what wasn’t possible before because of lack of bandwidth).

By the beginning of the last week, around the 30th of May, one backup copy job started to fail. This job contains all VM’s and should backup copy the data once a day to the secondary location. But one server, the main fileserver, failed with this error message:

Error: Block is not initialized. Failed to download disk. Reconnectable protocol device was closed. Failed to upload disk. Agent failed to process method {DataTransfer.SyncDisk}.

First i thought, ok, that can happen. No worries. After the second failure i did some checks and also opened a support case at Veeam support. If you don’t know how to collect log files for a Veeam support case, please have a look here.

First checks

Veeam support asked me to do some checks if the normal restore points of the failing VM are fine and with status “OK” in the repository.

To do so in Veeam, on the left hand side click “Backups > Disk > [Job / repo name]” and then right click this entry and choose “Properties”.

In my case all was fine, all restore points of this VM are with status “OK”.

Veeam asked me too if the regular backup to disk job for this VM runs successfully, and I could confirm that. I started also an Active Full backup for this source VM to make sure that all blocks are fine in the backup chain. After this Active Full i started the backup copy job too for this VM. But the error remained and this particular file server failed again.

Next steps

So basic VM backup runs fine, but backup copy of this VM still fails. The next step was to check the restore point for the copy, if they are “OK” or “incomplete”.

To do so in Veeam, on the left hand side click “Backups > Disk (copy)” and then right click this entry and choose “Properties”.

There i saw that the incremental restore points were incomplete. I’m not sure why, but they were. Probably the job screwed up at some point and nobody cared about it. Anyway. I had to fix that.

This was the first time i had to use the Veeam Backup Validator. With this CLI tool you can check and validate your backup files. My check failed or didn’t complete respectively, because the validator recognized that the files were incomplete.

I had to delete the restore points of this VM from the backup copy repository because they were incomplete. In Veeam Helpcenter you’ll find a good article about how to do that, about how to remove missing restore points.

Final steps

Now after all the restore points of this VM in the backup copy repository were deleted, the backup copy job runs and copies a complete full restore point from source to target repository.

Problem solved.

Personal thoughts

In my personal opinion, if there is an error or a warning, you don’t have to worry to much about that. It can happen. A service not started or crashed, a connection timed out, wrong credentials, or what ever reason. You can investigate and try to fix it. But if you know your customers infrastructure, so as i do, then you know also their specialities. If a Veeam job fails or brings a warning, i check the logs and then i know why it happened (at least most of the time). Probably it was my fault when i forget to set the correct credentials, or it’s one of the above reasons. Mostly i’ll give it a second try. When there is everything fine, then i’m fine too. If not, then i’ll dig into it and start with troubleshooting.

But as i said, that’s my personal opinion. You can truly switch to panic mode and start immediate actions with research and troubleshooting. But i don’t. Because i often don’t have time for such immediate quick shot actions, but i take my time to plan the necessary steps.

If this article helped you with troubleshooting, please let me know and drop a comment! Thanks!