! Please note that this is a snapshot of our old Bugzilla server, which is read only since May 29, 2020. Please go to gitlab.xfce.org for our new server !
Freeze new copy where source or target device is shared with another running one
Status:
RESOLVED: MOVED

Comments

Description Cyrille Pontvieux 2020-04-13 12:44:35 CEST
Created attachment 9715 
patch based on rename-on-fail-copy branch

patched based on rename-on-fail-copy branch.

A transfer job register the device id for the first file to copy (source and destination path).
Then when another transfer job is started, it checks if any of its source/target device id match another running job (it also depends on user preferences).
If yes, it will kind of freeze the job until all other jobs with matching device id are either finished, canceled or paused.
A frozen transfer job could also be forced to do the transfer with a GUI button.
Preferences for freeze on same source/target device or not have been added in Thunar Preferences, under Behavior tab.

The idea is to lesser IO pressure on devices when doing multiple copies/moves, especially on slow devices (USB, network, …)

The attached patched is based on my previous patch/bug (https://bugzilla.xfce.org/show_bug.cgi?id=16686)
The commit can also be found in a git branch (url attached to this bug).
Comment 1 Cyrille Pontvieux 2020-04-13 13:02:23 CEST
Created attachment 9718 
patch based on rename-on-fail-copy branch
Comment 2 Cyrille Pontvieux 2020-04-14 14:04:42 CEST
`gtk-media-play-ltr` and `gtk-media-pause` icon names seems to be ok. I found them in icons/gnome and they symlinks to `media-playback-start` and `media-playback-pause` respectively. Them or their symlinks could potentially be a good replacement.

Is there a way to find out a somehow standard, or de-facto standard, list of icons expected on a regular Linux or BSD system?
Comment 3 Cyrille Pontvieux 2020-04-14 14:08:10 CEST
Sorry my last comment was on wrong bugreport :-(
Comment 4 Reuben Green editbugs 2020-04-16 16:48:05 CEST
So this patch fails on my system.

>The attached patched is based on my previous patch/bug (https://bugzilla.xfce.org/show_bug.cgi?id=16686)
I have applied the patches from bugs 16685 and 16686 on top of the latest master (with "git am"), but running "git am" to apply this patch results in a failure:

error: patch failed: thunar/thunar-progress-view.c:622
error: thunar/thunar-progress-view.c: patch does not apply

Maybe there has been some change on master in the meantime? Or maybe I am doing something dumb :)
Comment 5 Cyrille Pontvieux 2020-04-16 19:32:09 CEST
Created attachment 9730 
New patch to apply against pause-on-copy patch only.
Comment 6 Reuben Green editbugs 2020-04-18 00:04:23 CEST
So this now applies, compiles and runs fine on my system. Thanks!

One possible bug: if I copy one file onto a device and then while the copy is running start another file copy, I see both jobs in the File Operations dialog with the first job running and the second one frozen with a "Force" button. Now I would expect that if I press the "Force" button, it should turn into a "Pause" button so that I can pause/resume the second job like I can with the first job. However, if I do press the "Force" button, it just stays as a "Force" button, and pressing it again has no effect, so that there is no way to pause the second job. If I let the first job complete then the second job starts and the "Force" button changes to a "Pause" button as expected.
Comment 7 Reuben Green editbugs 2020-04-18 00:07:23 CEST
Also Cyrille, is this related to bug #12123?
Comment 8 Cyrille Pontvieux 2020-04-18 01:51:25 CEST
Ah, this worked at some point, so I must have made a last-minute change that breaks it. I will investigate.

About the bug #12123, it's an other way of achieving the same goal. I did a proposal some years ago on this bug, and so I think the features are related even if resolved in another way.
Comment 9 Cyrille Pontvieux 2020-04-18 14:06:42 CEST
Created attachment 9743 
Patch against latest master

Fixed and some modification also applied regarding coding standards.
Comment 10 Reuben Green editbugs 2020-04-18 20:26:14 CEST
(In reply to Cyrille Pontvieux from comment #9)
> Created attachment 9743 
> Patch against latest master
> 
> Fixed and some modification also applied regarding coding standards.

Yep, this fixes the bug from comment 6. Thanks for the fix, Cyrille!
Comment 11 Cyrille Pontvieux 2020-04-25 19:56:58 CEST
https://gitlab.xfce.org/xfce/thunar/-/merge_requests/1
Comment 12 alexxcons editbugs 2020-05-10 22:32:10 CEST
*** Bug 12577 has been marked as a duplicate of this bug. ***
Comment 13 alexxcons editbugs 2020-05-10 23:26:13 CEST
I just played a bit with the new feature, very nice !

IMO it fixes what was requested in Bug #12123 / is a duplicate, though I dont want to close it as a dup. since there is a bounty of 100$ on it: https://www.bountysource.com/issues/25667178-feature-request-queued-file-transfer
... after pushing it, it's up to Beron to decide if Bug #12123 got fixed by it / you get that reward  :)

I am a bit unhappy with the preferences:
If I copy some files from my ssd to my ssd, I would like to have parallel copy as default --> untick both
If I copy some big files from my flashdrive to my ssd, I would need t tick "one transfer per source device" 
If I copy some big files from my ssd to my flashdrive, I would need t tick "one transfer per target device" 

So I would need to toggle these bits very often, whenever I copy many files to/from my USB drive. That's not very comfortable.

The author of Bug #12577 mentioned that he would like to have queued file transfer only for block devices ... while I am not so sure what exactly is a block-device, and what is not, I can get his point.

Possibly we could use "thunar_file_is_local" to identify if it makes sense to enable queued mode.
We could have a nice preferences setting with it by using an enum:

Transfer files in parallel:  [ Always | only for local files | Never ]
... actually the same like for "show thumbnails". You probably can copy the code.

You think that would make sense ?
Or possibly there is some gain to separate sources from targets, which I dont get ?
Comment 14 Cyrille Pontvieux 2020-05-17 22:32:38 CEST
Why would you want parallel copies from your ssd to your ssd? I don't see the point, as it will slow down a bit the whole copies.
And if you really want, you could hit the Force button. But maybe I don't understand something here.

About having two separate options in the parameters dialog for source and destination, I'm really not sure too. Maybe one option would be enough.

I can also maybe improve the code with using `thunar_file_is_local` but I need to be sure what a local file is for thunar. I will investigate the thumbnails code to see if it could indeed be of any help here.

For you're case (and probably for my personal laptop too), the « Only for local files » would be ok as a SSD is usally quick enough. But for my work laptop (mechanical disk), I will probably choose « Never ». And « Always » would be like the actual behavior.

I'll inform here when the branch will be updated accordingly (if technically possible).
Comment 15 Cyrille Pontvieux 2020-05-17 23:33:32 CEST
Well it's not that simple:
- `thunar_file_is_local` simply checks whether the scheme is `file://` or not. It just excludes any `http://`, and the like, schemes.
- `G_FILE_ATTRIBUTE_FILESYSTEM_USE_PREVIEW` is queried over the filesystem. But from what I checked with the thumbnail setting, it still builds thumbnails for any images I have on my USB key, despite I checked « only for local files ». That means that `G_FILESYSTEM_PREVIEW_TYPE_IF_LOCAL` is probably returned for my USB key even if it's not really « local ».

So I'm not sure that any of these settings will apply correctly to copy in sequences when using a slow USB key/disk.
It will probably work for copying over the network (like sshfs, nfs, …).

The checks for the device id still seams a better option.

The only drawback seems when doing multiple copying from and to a fast disk, like an internal SSD.
One way to fix this would be to allow parallel copies when source and destination device ids are the same.
Not sure how to display this as an understandable option…

By the way, I think that the thumbnail option might benefit from any finding here for what is really a « local file ».
Comment 16 Cyrille Pontvieux 2020-05-18 10:10:47 CEST
Using `g-file-find-enclosing-mount` and then `g-mount-can-eject`/`g-mount-can-unmount` could be an option to find out really *not local* filesystems (when advised as local by `thunar_file_is_local`) and hence probably slow device access.
Comment 17 Cyrille Pontvieux 2020-05-18 10:42:32 CEST
Ok I made some tests (fuse mounts, USB disk mounts, PTP mounts, MTP mounts, Samba mounts, and local files on different partitions).

`g_file_find_enclosing_mount` returns a non-NULL gmount only for attached devices ; for local files it returns NULL.
Then a test on `g_mount_can_unmount` could be done. Of course `thunar_file_is_local` should be done first.

I think by mixing all of these, the UX could be satisfactory. I will come with a new commit today or tomorrow with this idea.
Comment 18 alexxcons editbugs 2020-05-18 11:00:53 CEST
True, for my flash drives, as well thumbnails are generated :/  ...jepp, thumbnail settings could possibly profit from your planned changes :)

(In reply to Cyrille Pontvieux from comment #15)
> One way to fix this would be to allow parallel copies when source and destination device ids are the same.
Sounds like a good idea to me. I guess if there are multiple hdds/ssds, it would be ok to have queued file-transfer 

(In reply to Cyrille Pontvieux from comment #17)
> Ok I made some tests (fuse mounts, USB disk mounts, PTP mounts, MTP mounts,
> Samba mounts, and local files on different partitions).
> 
> `g_file_find_enclosing_mount` returns a non-NULL gmount only for attached
> devices ; for local files it returns NULL.
> Then a test on `g_mount_can_unmount` could be done. Of course
> `thunar_file_is_local` should be done first.
> 
> I think by mixing all of these, the UX could be satisfactory. I will come
> with a new commit today or tomorrow with this idea.
Promising !, thanks alot ! 

In case you want to as well check for "same device id", we probably would need an additional enum value .. e.g:
Transfer files in parallel:  [ Always | only for local files | only for local files if source device matches target device | Never ]
... hope that text would not be too long.
Comment 19 Cyrille Pontvieux 2020-05-21 16:47:16 CEST
Ok this is now done and tested (on my part).

I pushed another commit and rebased on master.

Please tell me what you think. If anyone can also test with any type of mounted device and any transfer mode (always parallel, parallel for local files, parallel for local files on same devices only, never parallel), it could help to possibly find an unwanted behavior in a corner case.
Comment 20 alexxcons editbugs 2020-05-24 09:35:38 CEST
Thanks alot for the changes, very nice !

I like the new enum, think it is more user-friendly now.
You can add a tooltip in order to explain the details (e.g. that the copy will be started and directly paused)

I tried different copy/setting combinations, think I found one case which needs improvement:
Started two copies from my SSD to a flashdrive, while having "only for local files" selected.
They got copied in parallel.
The file uris, if that may help to reproduce:
Source (local ssd):
/home/schwinn/software/thunar_test/Test1
/home/schwinn/software/thunar_test/Test2
Destination (USB flashdrive):
/media/schwinn/USB_STICK/Test1
/media/schwinn/USB_STICK/Test2

Re-tested on some other flashdrive, same result:
/media/schwinn/00BCB5F7BCB5E6F6/

On ething to mention: The other way around (copy from flashdrive to ssd) work well ! The second transfer is frozen.

... just as a side-node to ease testing. Creation of testfiles:
fallocate -l 1G bigfile1
Comment 21 Cyrille Pontvieux 2020-05-26 21:42:56 CEST
Thanks a lot for the testing.

I found the reason for the corner case and it should now be fixed in the latest version of the branch.

Thanks for fallocate tip, I was doing this with dd but it was a bit tedious.
Comment 22 Git Bot editbugs 2020-05-26 23:50:29 CEST
-- GitLab Migration Automatic Message --

This bug has been migrated to xfce.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.xfce.org/xfce/thunar/-/issues/294.

Please create an account or use an existing account on one of our supported OAuth providers. 

If you want to fork to submit patches and merge requests please continue reading here: https://docs.xfce.org/contribute/dev/git/start#gitlab_forks_and_merge_requests

Also feel free to reach out to us on the mailing list https://mail.xfce.org/mailman/listinfo/xfce4-dev

Bug #16688

Reported by:
Cyrille Pontvieux
Reported on: 2020-04-13
Last modified on: 2020-05-26
Duplicates (1):
  • 12577 Serialize operations when copying from/to the same device.

People

Assignee:
Xfce Bug Triage
CC List:
4 users

Version

Attachments

patch based on rename-on-fail-copy branch (31.32 KB, patch)
2020-04-13 12:44 CEST , Cyrille Pontvieux
no flags
patch based on rename-on-fail-copy branch (31.91 KB, patch)
2020-04-13 13:02 CEST , Cyrille Pontvieux
no flags
New patch to apply against pause-on-copy patch only. (31.92 KB, patch)
2020-04-16 19:32 CEST , Cyrille Pontvieux
no flags
Patch against latest master (32.49 KB, patch)
2020-04-18 14:06 CEST , Cyrille Pontvieux
no flags

Additional information