Copy file with target name with diacritics

[FMuro]

Take a PDF file named Gomez.pdf and pass the command cp Gomez.pdf Gómez.pdf. Then opening the new Gómez.pdf is impossible. It’s apparently corrupted. If you then pass cp Gómez.pdf Gamez.pdf the file Gamez.pdf can be opened again. I work heavily with diacritics because I have many PDF files with student names (mostly Spanish names). Thanks in advance for your help. Please, tell me if I can help debugging.

[holzschu]

I take diacritics (and everything that isn't 7-bit ASCII English in general) very seriously.
But I cannot reproduce this issue: I tried cp vc.pdf Gómez.pdf and I could open Gómez.pdf with both the view command and the Files app.

So there must be something different between our two settings, and we need to find what it is. It could be:

• the directory where the PDF file is (I tried ~/Documents and ~cloud).
• the command or application you use to view the PDF file.
• the fact that the cp command somehow corrupted the new PDF file (try diff Gomez.pdf Gómez.pdf, or ls -l G*mez.pdf).
• the system language (in Settings->General->Language & Region, mine is set to English) (at this point I'm grasping at straws)

[FMuro]

Thanks for your quick answer! Let me address your points.

• I’ve tried ~/Documens and a picked up folder, both fail.
• I’ve tried Preview, PDF Expert, and open choosing either afterwards.
• diff doesn’t return anything, ls returns the same data for both. I don’t think it’s a real corruption because the file can be opened if I rename it without diacritics.
• Spanish first, and some other preferred languages.

This is intriguing.

[holzschu]

You're absolutely right about the corruption issue, we can rule that out.

I switched my iPad to Spanish. The Gòmez.pdf filed created previously could still be opened, but when I created a new one (with cp Gomez.pdf Gómez.pdf), then that new file could not be opened with open. I can still visualize it with view, though, so it must be something related to open.
Switching back to English, the file I created while the iPad was in Spanish still cannot be opened. The error message (I assume you have the same?) is "You don't have permission to save the file "Gòmez.pdf" in the folder ...".
I can also visualize that file in the Files app, but not open it in PDF Viewer, which is very strange.

[holzschu]

Since the problem also happens in the Files app, then the issue is probably with the file name itself, not with the open command.

[FMuro]

I haven’t seen any error message. Opening fails silently in all cases for me. view doesn’t work in my case and switching the app to English doesn’t work either. I’ll switch the whole machine to English later, I can’t reboot right now. If there’s anything else I can do please tell me. Thanks a lot for your debugging effort!

[holzschu]

I think "PDF Viewer" is the only one that displays an error message (and only when called from the Files app, I used Files app -> preview -> Open in PDF Viewer).
I am going to check if the issue is with "cp" or with the file name itself by creating a Gòmez.pdf file directly. This is very puzzling.

[holzschu]

Reporting my results here because it helps me in debugging this issue:

• it doesn't happen (for me) if the file is in a-Shell's home directory or a subdirectory of it, but definitely happens if the file is in the iCloud folder.
• files created (as opposed to copied) have the same issue, so the problem is with the file name, not the creation method. Even an empty file created with touch has the issue.

So we have: problems come from the file name containing diacritics (or at least "ò") and not being inside the main Sandbox.

[holzschu]

I've asked the question on Apple's Developer Forum: https://developer.apple.com/forums/thread/817449

[holzschu]

Apple engineers came back with an explanation and a solution: https://developer.apple.com/forums/thread/817449?answerId=878470022#878470022

Short version: it's an iOS bug. It's related to the existence of two different forms for UTF8 characters: decomposed (NFD) and composed (NFC). Our non-english keyboards produce the composed form, and the system expects the decomposed form.

I have changed all the places with a-Shell calls a Unix command to first convert the command into the decomposed form. Now, all files created by a-Shell will use the decomposed form. This change will be in the TestFlight version build 534 (which should be online tomorrow, Friday 6th) and in the AppStore version 2.0.0.

I cannot do anything for the files created before the change, but I think you will be able to rename them.

Thank you so much for reporting this issue and helping me with fiding a way to reproduce it.

[holzschu]

Update: the TestFlight version is available: https://testflight.apple.com/join/WUdKe3f4 for a-Shell.

If it's your first time using TestFlight, you will an alert "you already have this app installed, you could lose all your data". It's a standard warning, in this case you won't lose all your data.