T-rev's Blog

44 2014-08-12 15:29 Robocopy script problem with flaky NIC

I just encountered a crazy data loss problem that had me mystified and questioning my core computing beliefs. Some of my recently modified or recently created files were being wiped to zero file size.

Specifically, the affected filename was all that remained, with a last-modified date more recent than when I had actually last modified the file. Some of my important records were being wiped out.

I have multiple copies of backups on different servers, but my strategy is mostly geared toward protecting against disk crashes or corruption, and I wasn't maintaining any file version history. Thus, if I didn't notice instances of the zero-file-size problem in time, the "newer" empty file could propagate through my backup copies, with the empty file being written over the other backup copies of the file.

I'm now pretty sure I've figured out the problem. It's a combination of a flaky gigabit ethernet port in my (9-month-old) Toshiba laptop and the structure of one of my backup scripts, which runs robocopy first in one direction and then in the other.

The flaky connection would cause the first robocopy call to fail in mid-stride. The first robocopy call at this point had already created a placeholder filename in the destination location, with the timestamp set (meant to be temporarily) to current time, but had not copied the data and actual timestamp. Subsequent to this, the second robocopy call then copies the empty placeholder back, over-writing the data in what should have been the source file!

Obviously I need to implement version history backup.

Another measure I've now taken is telling robocopy to ignore files of zero size. I added the option /MIN:1, which will exclude files smaller than 1 byte. The only (minor) downside of this is that I have some information contained in filenames of intentionally empty files, but I can get around that by giving those files some trivial content.


No comments yet

Add Comment

This item is closed, it's not possible to add new comments to it or to vote on it