作者: bongbong3481 時間: 2023-11-24 15:13 標題: Robocopy 拖慢部機？
本帖最後由 bongbong3481 於 2023-11-25 14:56 編輯
公司有個sharefolder（大約40G 左右，好多細FILE，EXCEL 同埋OUTLOOK EMAIL FILE 多，FILENAME繁簡英都有），每日系自己部機起個日期FOLDER ，然後用ROBOCOPY 將SHAREFOLDER COPY 去我部機［部機有2個disk，一個裝window及軟件，一個專門backup data］，一行ROBOCOPY 部機就有D 慢，吾係好順。/R/W/MT 已經SET 到好細，都沒咩用。
請問點解決？
作者: s84292 時間: 2023-11-24 17:55
本帖最後由 s84292 於 2023-11-24 18:04 編輯
公司有個sharefolder（大約40G 左右，好多細FILE，EXCEL 同埋OUTLOOK EMAIL FILE 多，FILENAME繁簡英都有） ...
bongbong3481 發表於 2023-11-24 15:13
ROBOCOPY你預實慢架
佢OS 核心同IO都要大負載
一係你 limit ROBOCOPY 個bandwidth
在ROBOCOPY 之後加 /ipg:750
750呢個數字可以自己改,試到你岩為止
可以試唔落/MT, 由得WINDOWS自己行為預設數(應該為8)
一係就改用DFS replication
作者: s20012797 時間: 2023-11-24 22:43
本帖最後由 s20012797 於 2023-11-24 22:51 編輯
Based on your description, it seems like the issue is related to the size of the files being copied and the number of files being copied. Here are some possible solutions that you can try:
Increase the number of threads: You can try increasing the number of threads used by Robocopy by adding the /MT option followed by the number of threads you want to use. For example, /MT:16 will use 16 threads. This can help speed up the copying process by allowing Robocopy to copy multiple files simultaneously .
Disable logging: Robocopy logs every file that it copies, which can slow down the copying process. You can disable logging by adding the /NFL and /NDL options to your command. For example, /NFL /NDL will disable logging of file and directory names .
Use unbuffered I/O: You can try using unbuffered I/O by adding the /J option to your command. This can help speed up the copying process for large files .
Disable Windows Copy Offload: You can try disabling Windows Copy Offload by adding the /NOOFFLOAD option to your command. This can help speed up the copying process for large files .
it is possible to use Python scripts as an alternative to solve the problem. Python provides various libraries and modules that can help with file operations and copying files.
One option is to use the `shutil` module in Python, which provides a high-level interface for file operations. You can use the `shutil.copytree()` function to recursively copy a directory and its contents to another location. Here's an example:
python:
- import shutil
- source_folder = 'path/to/source/folder'
- destination_folder = 'path/to/destination/folder'
- shutil.copytree(source_folder, destination_folder)
You can also use the `os` module in Python to perform file operations. The `os.listdir()` function can be used to get a list of files and directories in a given folder, and then you can use the `shutil.copy()` function to copy individual files. Here's an example:
python:
- import os
- import shutil
- source_folder = 'path/to/source/folder'
- destination_folder = 'path/to/destination/folder'
- for file_name in os.listdir(source_folder):
- source_file = os.path.join(source_folder, file_name)
- destination_file = os.path.join(destination_folder, file_name)
- shutil.copy(source_file, destination_file)
Using Python scripts gives you more flexibility and control over the copying process. You can customize the script according to your specific requirements and handle any additional operations or conditions as needed.
Remember to adjust the file paths (`source_folder` and `destination_folder`) in the examples to match your specific setup.
作者: bongbong3481 時間: 2023-11-25 06:32
回覆 3# s20012797
我臨時用左python copy tree 頂住先，但robocopy 有太多功能選項玩，prefer robocopy （我估用python 都模擬到，但要寫好多code ,自己懶，沒辦法）
作者: tgo 時間: 2023-11-25 07:13
你部機乜嘢config? 我都有咁用，完全冇問題。你copy緊開埋個task manager睇埋個cpu/hdd 扯到幾高。
公司有個sharefolder（大約40G 左右，好多細FILE，EXCEL 同埋OUTLOOK EMAIL FILE 多，FILENAME繁簡英都有） ...
bongbong3481 發表於 2023-11-24 15:13
作者: java2 時間: 2023-11-25 10:58
本帖最後由 java2 於 2023-11-25 11:01 編輯
有D 慢即係點慢? 日日 copy 40G files 去你部機HDD?
為乜? 如果係Backup 應用incremental copy
公司有個sharefolder（大約40G 左右，好多細FILE，EXCEL 同埋OUTLOOK EMAIL FILE 多，FILENAME繁簡英都有） ...
bongbong3481 發表於 2023-11-24 15:13
作者: bongbong3481 時間: 2023-11-25 11:10
你部機乜嘢config? 我都有咁用，完全冇問題。你copy緊開埋個task manager睇埋個cpu/hdd 扯到幾高。
...
tgo 發表於 2023-11-25 07:13
8核CPU，16G 行win10，用TASK MANAGER 睇，
CPU 扯得吾高，得幾個%，反而HARD DISK 去到100%
作者: bongbong3481 時間: 2023-11-25 11:15
回覆 6# java2
其實最快是用incremental copy， 不過有d file 改完，想還原番去前幾個版本就有問題。所以每日起個date folder 將成個sharefolder backup
作者: java2 時間: 2023-11-25 13:20
用你自己部機HDD 做backup? 日日寫40G? 慢都正常, HDD I/O 喎. 你唔覺呢個方法本身就有問題?
回覆 java2
其實最快是用incremental copy， 不過有d file 改完，想還原番去前幾個版本就有問題。所以每 ...
bongbong3481 發表於 2023-11-25 11:15
作者: bongbong3481 時間: 2023-11-25 15:06
回覆 9# java2
其實copy 速度慢都沒所謂，最大問題系令到成部機都好慢，影響做野。 我部機有2個disk，一個裝window及軟件，另一個backup data，已經分開disk。
（其實我都有用7zip 黎backup 個sharedrive，速度都好快，不過吾知咩原因，成日話有d file 搵吾到，所以成個folder 去copy。 另外，夜晚又要backup outlook archive，沒辦法夜晚做 ）
作者: tgo 時間: 2023-11-26 00:18
robocopy 可以淨係 copy modified files，你可以照舊每天開個新folder，淨係copy modified files，咁會抄少好多files, 唔得再好似上面師兄講 set /ipg, MT set 做 1。
回覆 java2
其實最快是用incremental copy， 不過有d file 改完，想還原番去前幾個版本就有問題。所以每 ...
bongbong3481 發表於 2023-11-25 11:15
作者: bongbong3481 時間: 2023-11-26 06:40
回覆 11# tgo
淨係copy modified file是快的，都可以試下。不過要搵番d file， 就有少少吾方便，因為每日一個date folder ，如果click 入去遂個搵，就花d 時間，只能search file name。
作者: tgo 時間: 2023-11-26 11:11
你係root folder，search 嗰個 file name，佢會搵晒全部sub folder，list晒全部出黎。
就算你而家咁做法，你一樣要逐個date folder入去睇。
回覆 tgo
淨係copy modified file是快的，都可以試下。不過要搵番d file， 就有少少吾方便，因為每日一 ...
bongbong3481 發表於 2023-11-26 06:40
作者: s20012797 時間: 2023-11-26 14:54
你係root folder，search 嗰個 file name，佢會搵晒全部sub folder，list晒全部出黎。
就算你而家咁做法， ...
tgo 發表於 2023/11/26 11:11
用powershell 寫個腳本裝copy左的modified file資料寫番入txt當log己經得
作者: tgo 時間: 2023-11-27 10:47
robocopy 本身都有option寫log。
用powershell 寫個腳本裝copy左的modified file資料寫番入txt當log己經得
s20012797 發表於 2023-11-26 14:54
作者: s20012797 時間: 2023-11-27 10:58
robocopy 本身都有option寫log。
tgo 發表於 2023/11/27 10:47
powershell主要作用係比較log,
btw,多謝ching提醒
作者: epcsub 時間: 2023-11-29 20:56
用tree 呢個command line utility 去gen 一個directory map (text檔 show到directory structure)
每逢backup完 就做一個
唔好search file 喺個map到搵
via HKEPC Reader for Android