Page MenuHomePhabricator

Performance Issues
Closed, ResolvedPublic

Description

Aaron asked on FB:

Does it now support faster than 100mb speeds? Only reason I stopped using it

Replicator uses Win32 ReadFile and WriteFile from Kernel32.dll. Plus, there's a call to check the free space on the destination file after every file is checked.

Public Const API_BUFSIZ = 2 ^ 16

This is a 64k block size. Considering the speed of today's hardware, try just upping this to the maximum Win32 will accept. If that's all good, then perhaps set the block size based on how quickly the copy callback is being called. The point of the callback is UI, anyway...

Event Timeline

Joe created this task.Dec 8 2018, 5:35 PM
Joe created this object in space S5 Public.
Joe triaged this task as High priority.
Joe created this object with visibility "Public (No Login Required)".
Joe updated the task description. (Show Details)Dec 8 2018, 5:38 PM
Joe added a comment.Dec 9 2018, 8:20 AM

This is probably a duplicate of T33: Faster, Faster, Faster

Joe added a comment.Jan 6 2019, 1:53 PM

Not updating display for every file being compared, so T75: Improve progress indication relates here.

Joe added a comment.Jan 6 2019, 3:57 PM
This comment was removed by Joe.
Joe added a comment.Jan 6 2019, 4:20 PM
This comment was removed by Joe.
Joe added a comment.Jan 6 2019, 5:43 PM

Enlarging the buffer seems to hurt performance on huge files. Reducing it seems to cause system weirdness on huge files. Investigate more tomorrow.

Joe added a comment.Jan 7 2019, 10:37 AM

CopyFileEx now in place and working. 134 MB/s to USB 3.0 drive instead of 8.

Job: BigFiles
Run Time: 1/7/2019 10:10:50 AM to 1/7/2019 10:12:52 AM (Duration: 2:01)
Status: Completed Successfully
Files Copied: 7
Bytes Copied: 16,309,209,088
Bytes per Second: 134,595,526

Karen already had 99.9% of the work done

She'd commented out the call to CopyFileEx version of her own code -- I added the delete before copy and test for read lock stuff to it.

Now I'm looking for the reason she didn't use it, or had removed it. By looking, I mean testing for problems.

Now using Win32 api CopyFileExW, so this might fix the long pathname problem described in T29: Super-long Pathnames

Joe added a comment.Jan 7 2019, 6:11 PM

This happens:

1/7/2019 6:07:03 PM: Error: Copy of K:\_VISTA_\D\VB6\Replicator\BigFiles\bigbase.iso to Y:\Karen's Tests\BigFiles\bigbase.iso Failed (Error 1235 - The request was aborted.):
Joe added a comment.Jan 7 2019, 6:32 PM

The 1235 Error only happens when running debug from VB's IDE.

Joe closed this task as Resolved.Jan 7 2019, 7:21 PM
Joe claimed this task.

Doesn't happen from VB6 IDE when copying to an internal drive. My external USB drive is encrypted. Maybe that has something to do with it. ANYWAY, not able to reproduce it with a live-compiled version of the program.