Missing Files – Forumite.site

This topic has 24 replies, 5 voices, and was last updated 5 years, 2 months ago by Ed P.

Viewing 20 posts - 1 through 20 (of 25 total)

1 2 →

Author

Posts
December 8, 2020 at 9:53 pm #64438
Drezha
Participant
@drezha
Forumite Points: 0
Found that a load of files had been deleted off my machines today.

I keep a manual backup of my files at work, updating via SFTP with Freefilesync. Anyhow, it tried to propogate some deletions today of files I didn’t delete myself. Thankfully I was able to restore all from backup but it gave me cause for concern.

Checking my Synology Hyper Backup logs, it seemed it happened on the 1st December between my backups at 16:30 and 22:30, but I don’t recall doing anything that might have deleted these then. Checking Resilio Sync logs isn’t helpful, as they only store the logs on the machine itself, so I’d have to check each and every one individually.

I’d been considering switching to Synology Drive Sync instead, as it works where I’d found occasions where Resilio doesn’t (namely at work on the Eduroam network, as Resilo is based on torrent syncs and that doesn’t work on Eduroam). This looks like it may speed up my move! At least the server would keep the records of when the deletions happened and when (and I think it would also tell me which client caused it as well). I’ll have to see if it works on the M1 Mac. And I do seem to recall possibly not being able to edit the files directly on the server via SMB as well, but I’ll have to check that, as that might be an issue (unless the Drive client appears a location in the iPad Files app).

Anyhow, I was lucky today!

"Everything looks interesting until you do it. Then you find it’s just another job" - Terry Pratchett
December 9, 2020 at 6:56 am #64442
Ed P
Participant
@edps
Forumite Points: 39
One possibility is Windows, if it thinks it is short of space it will delete files without asking. I suspect that if it has been allowed to ‘see’ network files it will do the same:

link

You may want to check if Apple have the same malign policy!
December 9, 2020 at 8:13 am #64444
Dave Rice
Participant
@ricedg
Forumite Points: 7
Synology Drive is getting quite sophisticated now. Since W10 1809 you can have on demand sync, the same as One Drive. Since 20H2 Storage Sense is now aware of it and can be set to do some space saving automatically if you run low locally.
December 9, 2020 at 12:31 pm #64452
Drezha
Participant
@drezha
Forumite Points: 0
Ed P wrote:

One possibility is Windows, if it thinks it is short of space it will delete files without asking. I suspect that if it has been allowed to ‘see’ network files it will do the same: link You may want to check if Apple have the same malign policy!

There are all local files, that downloaded using Resilo, rather than on the network so that could be the case. However, I have about 75% free space still on the Windows machine and the similar on the Mac. Storage sense is off on my laptop, I’d have to check the desktop.

Only thing I can think of is that the Mac one is stored on an external drive and the at one point the drive was disconnected. However, when that occurs, Resilo normally just shuts down and refuses to sync, which is what appeared to happen.

I’ll have to keep an eye on things over the next few days.

"Everything looks interesting until you do it. Then you find it’s just another job" - Terry Pratchett
December 9, 2020 at 1:06 pm #64454
keith with the teef
Participant
@thinktank
Forumite Points: 0
Don’t forget tools like Recuva. I paid for a license one time but the enhanced feature set was not really better than the free version.
December 23, 2020 at 8:31 am #64845
Drezha
Participant
@drezha
Forumite Points: 0
Hum, something funny might be going on – I found a courrpted file last night on my gaming rig as well (a JPG).

I think I’ll be doing some drive checks on all the machines to see what’s happening. Currently running Memtest on my Synology – it also turns out that I’m not getting the benefit of the BTRFS formatting on there, as I’m not running in RAID (so I’m not getting the repairing corrupted files automatically). That wasn’t made clear when I started :negative:

"Everything looks interesting until you do it. Then you find it’s just another job" - Terry Pratchett
December 23, 2020 at 8:51 am #64848
Dave Rice
Participant
@ricedg
Forumite Points: 7
I didn’t know that, just read up about it. Synology layers BTRFS on top of mdadm (Linux software RAID) and BTRFS RAID isn’t stable enough yet. BTRFS does the checksumming and bit rot detection, md provides the redundancy.

I always use some sort of RAID, usually SHR, so doesn’t apply. Having owned a + box for a while now I am convinced the extra £ are worth it, mostly for the Business Backup suites. The ability to backup Teams and Workspace for my customers (in real time) is invaluable and the bare metal PC backup with automatic deduplication is working well.
December 23, 2020 at 10:01 am #64851
Drezha
Participant
@drezha
Forumite Points: 0
Yeah I went back and read the BTRFS page on Synology site and it’s clearer that I need SHR or RAID to make use of it – I’d basically seen a comment on Reddit today about BTRFS performance on the new Linux kernel, so I did some digging then.

File was corrupt on the gaming PC but I didn’t check the NAS to see if it had propagated, just downloaded a working file from Synology C2 and replaced.

We’ll see what memtest comes up with and it may mean the purchase of another 8TB drive to create a SHR pool.

"Everything looks interesting until you do it. Then you find it’s just another job" - Terry Pratchett
December 23, 2020 at 2:09 pm #64858
Drezha
Participant
@drezha
Forumite Points: 0
Ordered another 8TB drive from Amazon for delivery tomorrow, so I’ll be reformatting everything to put it in to SHR. Almost ordered a DS420+ to replace it with instead!

I’ll double up on my offsite and non Synology backups if there are further issues.

"Everything looks interesting until you do it. Then you find it’s just another job" - Terry Pratchett
December 23, 2020 at 3:12 pm #64862
Ed P
Participant
@edps
Forumite Points: 39
I had no idea what was meant by the acronym SHR so I looked it up, then spent two minutes wondering what the hell was a Spontaneously Hypertensive Rat. I did eventually find that the S stood for Synology!
December 23, 2020 at 3:21 pm #64865
Dave Rice
Participant
@ricedg
Forumite Points: 7
I added an 8GB sodimm to my D218+ with no problems. I just made sure the specs were the same, got that off a You Tube somewhere.

I’ve got all my customers on C2 now, after the initial backup it’s processing most daily backups within 10 minutes.

DSM 7 on the rackstation is tootling along with no dramas. The Hybrid Cloud features look good (endless storage) but need BTRFS which I can’t do on the RS. That RS chassis has been in use 24/7 for 5 years+ (new drives recently) and just shows what value these boxes are. It’s only a 1.3Ghz dual core ARM with 1Gb ram.
December 23, 2020 at 3:56 pm #64869
Drezha
Participant
@drezha
Forumite Points: 0
I think I’ve got a 4GB stick in there to match the one already in place, for a total of 6GB. At the minute, I’m not over 1GB, so it’s not really being taxed to much just yet!

C2 is fine – currently on the lowest tier, but may look at moving to the higher level in the future. However, all the major items I want at the minute can be protected by the smaller 300GB tier (that’s all mine and my wife’s photos (65GB) and all my documents (less than 20GB). Other files are thrown in there. The only thing I dislike about it is I can’t remove an item that is no longer required. Yes, I can stop it backing up to start with, but once it’s up there, I can’t delete all instances of it at all. :negative:

"Everything looks interesting until you do it. Then you find it’s just another job" - Terry Pratchett
December 23, 2020 at 9:25 pm #64873
Dave Rice
Participant
@ricedg
Forumite Points: 7
I’m on 2TB now, the surveyors take 100+ photos on every report and even reduced to 1080P (which took me ages to convince them to do) consumes 200GB just in the working set. There’s a 700GB archive (backed up locally, twice) that they are working through shrinking the photos that will end up in C2 too, considerably reduced in size.

They are the ultimate client for Hybrid Storage when their 3TB runs out, but how do you back up the in the Cloud files? Once DSM 7 is released I’ll be upgrading the 218+ and finding out these things but unfortunately not a zero cost (the beta gives you 1TB for free).

With regards deleting files from the backups, surely if you’ve set a retention policy they will eventually drop out? I can see occasions when retention periods wouldn’t be a good thing though, but that would really be for archives which I do as one offs.

I’m certainly not regretting choosing Synology over Qnap, it was a 50 / 50 decision, and the roadmap for the future looks good. I’m going to start evaluating Synology Office as a collaboration tool as Google Workspace has convinced me that you do not need Microsoft, indeed Teams is getting ridiculously complicated. MS are turning it into the fully fledged Domain in the Cloud but bringing all the legacy with them. You need to understand all that MS terminology.
December 24, 2020 at 1:00 pm #64894
Drezha
Participant
@drezha
Forumite Points: 0
Dave Rice wrote:

With regards deleting files from the backups, surely if you’ve set a retention policy they will eventually drop out? I can see occasions when retention periods wouldn’t be a good thing though, but that would really be for archives which I do as one offs.

I can’t set rotation on the cheapest C2 cloud backup. The website states:

Plan I protects your data through daily backups with a default retention policy. C2 Storage for Hyper Backup will retain 11 backup versions (one latest, 7 daily, 3 weekly) going back 30 days.

However, my latest backup is still showing a folder that I no longer want included within the storage and seems to be backed up nightly.

"Everything looks interesting until you do it. Then you find it’s just another job" - Terry Pratchett
December 24, 2020 at 3:50 pm #64901
Dave Rice
Participant
@ricedg
Forumite Points: 7
Ah OK. I moved off Plan 1 ages ago. Looks like you’ll have to wait 30 days then it’ll drop out.
December 24, 2020 at 4:34 pm #64907
Wheels-Of-Fire
Participant
@grahamdearsley
Forumite Points: 4
A slight deviation, but still file and backup based.

I just noticed that the desktop I look at all day on my Surface pro 7 is actually C:\users\”Me”\Onedrive\desktop.

I had wondered how my desktop was getting backed up to Onedrive but I never really looked before.
December 26, 2020 at 5:51 pm #64931
Drezha
Participant
@drezha
Forumite Points: 0
Dave Rice wrote:

I didn’t know that, just read up about it. Synology layers BTRFS on top of mdadm (Linux software RAID) and BTRFS RAID isn’t stable enough yet.

That bit me in the backside today. I’d copied the items I wanted to my BTRFS formatted 3TB hard drive, then removed it, put the new 8TB in and reformatted the lot. Put the 3TB in an external enclosure and attached to the NAS – not recognised.

Attached to a Raspberry Pi, not recognised. In the end, Fedora 33 (the latest Fedora, which uses BTRFS as normal) managed to pick it up. Here it’s clear that Synology create a RAID 1 array with a single disk for BTRFS, which is why the disk didn’t pick it up separately.

Now it’s currently transferring it’s data across a USB connection at 20MB/s so its taking a lot longer to move across than expected.

However, thankful the data is still intact – I know for next time to move it to an ext4 formatted drive direct on the NAS or export over the network to any other drive plugged in. :negative:

"Everything looks interesting until you do it. Then you find it’s just another job" - Terry Pratchett
December 26, 2020 at 9:54 pm #64933
Dave Rice
Participant
@ricedg
Forumite Points: 7
I think that’s what SHR does so you can add a drive later without having to build an array. As you (& I) have found, out one of the best ways to transfer data is to an unencrypted external EXT volume. A Hyper Backup single version job is good way of doing it.

I put 3 x 3TB in the RackStation (leaving a slot free) but decided to go with RAID 5. The reason being I want to use that spare slot to muck about with things like iSCSI without having to touch any volume with data on it. All EXT4 as it’s not an Intel CPU so no BTRFS. 6TB + some for experimentation will do me just fine. I have plenty of old but not quite knackered spinners hanging about.

The plan now is for the RackStation to take over the general day-to-day duties of a backup target, ISO download server and general F&P leaving the DS218+ to do the things that only a +box can or does better (like on-the-fly video transcoding).

Between them that’s 9TB of storage, should be enough for a while. Our first NetWare server only had 8GB (that never got anywhere near full) that ran the Post Office South Wales & South West Region for over 5 years before NT took over. It serviced nearly 200 PCs in the end with data increasingly being downloaded from mainframe / AS400 platforms for analysis in Windows spreadsheets. 8GB would probably last a morning now, if that. I’ve seen swap files bigger.
December 26, 2020 at 10:13 pm #64935
Drezha
Participant
@drezha
Forumite Points: 0
Dave Rice wrote:

I think that’s what SHR does so you can add a drive later without having to build an array.

From memory, I didn’t set it up as an SHR volume to start with, as it was a 8TB and 3TB drive in there and each drive was set up separately.

I sold the old Synology the 220+ replaced, so no fancy backup targets for me.

The 8TB should be sufficient for a while I hope, otherwise I’ll either have to upgrade the drives or get a 4 bay NAS! At the minute, I don’t think I’m really using the high end nature of the + models and probably would have been fine with the DS218. However, I guess it’s future proof for now.

I did consider getting a QNAP to see what they could bring to the table but the cheapest ones are probably like the cheapest Synology ones and to probably be avoided. My dad is using my old DS116j (or was it DS115j) though and still finds it fine – though he’s using it purely for storage I think.

"Everything looks interesting until you do it. Then you find it’s just another job" - Terry Pratchett
December 27, 2020 at 3:52 pm #64946
Dave Rice
Participant
@ricedg
Forumite Points: 7
It’s really splitting into two product lines, the + with their Intel CPUs for business and the ARM for more basic needs. Being able to host the PiHole as a VM really has been useful.
Author

Posts