linux – Differences in folder size as a result of copying folders using rsync

Question: Question:

When I copy the entire folder to another server using rsync, I am in trouble because of the difference in size. If anyone finds a solution, I would like to ask a professor, and I am asking a question here.

Let me explain the situation a little.

I am currently using two servers (let's call them A and B).
Both operating systems are set to ubuntu 16.04.

I want to copy a folder on server A to server B.
The folder contains multiple files as shown below.

Nov132018
├── test.AQ863.1.1.h264
├── test.AQ863.1.2.h264
├── test.AQ863.1.3.h264
├── test.AQ863.1.4.h264
└── test.AQ863.1.5.h264

I copied this entire folder to Server B using the following command:

rsync -av -e ssh /media/user1/1C3C-F8AC/Nov132018 amori@ipaddress:/tank/user1/tmpNAS/NAS1/movies

When I checked the transferred folders on the server B side, all the files including all the folders were transferred.

However, the problem was that the folder size was 20 times larger.

In particular,
On the server A side

$ ls -lh Nov132018/
total 504M
-rw-r--r-- 1 user1 user1 504M Nov 13 04:16 test.AQ863.1.1.h264
-rw-r--r-- 1 user1 user1    0 Nov 13 04:26 test.AQ863.1.2.h264
-rw-r--r-- 1 user1 user1    0 Nov 13 04:36 test.AQ863.1.3.h264
-rw-r--r-- 1 user1 user1    0 Nov 13 04:46 test.AQ863.1.4.h264
-rw-r--r-- 1 user1 user1    0 Nov 13 04:56 test.AQ863.1.5.h264

It was,

On the server B side

$ ls -lh Nov132018/
total 10G
-rwxr-xr-x 1 amori amori 504M Nov 13 04:16 test.AQ863.1.1.h264
-rwxr-xr-x 1 amori amori    0 Nov 13 04:26 test.AQ863.1.2.h264
-rwxr-xr-x 1 amori amori    0 Nov 13 04:36 test.AQ863.1.3.h264
-rwxr-xr-x 1 amori amori    0 Nov 13 04:46 test.AQ863.1.4.h264
-rwxr-xr-x 1 amori amori    0 Nov 13 04:56 test.AQ863.1.5.h264

was.

Even though the size of the transferred files is correct, the folder size is very different between 500M and 10G.

I don't understand why such a problem is occurring.
I also want to keep it to 500M on the server B side.
If anyone knows such a method, please teach me.


Now that I know the cause, I'll fill in some additional information.

Apparently the cause is block size.
If you use the df command to find out,
Server A's block size (1K-blocks) is 60049056
The block size (1K-blocks) of server B is 10978791680
was.

There is other data on server B as well, so I don't know what happens if I change the block size, so I'm stopping here. If you have any advice, I would appreciate it if you could let me know.

Answer: Answer:

Maybe it's because I copied the sparse file. Try copying rsync again with the --sparse option.

Please delete the copy destination file before trying again. Because rsync doesn't see any difference in sparse state in sync (recopy) decisions.

Scroll to Top