Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Warning: Detected unknown filetype (descriptor with ext)" #143

Open
steilerDev opened this issue Jan 4, 2023 · 29 comments
Open

"Warning: Detected unknown filetype (descriptor with ext)" #143

steilerDev opened this issue Jan 4, 2023 · 29 comments
Labels
class(known issue) A known issue that might not be easy to fix or needs additional input. status(help needed) Help from other people is necessary to resolve this issue

Comments

@steilerDev
Copy link
Owner

steilerDev commented Jan 4, 2023

The list of supported file types need to be hardcoded.

There is no logic behind supporting additional extensions, besides knowing how the filetype (as returned by the Apple backend) is mapped to a file extension.

If you should come across an error with root cause Unknown filetype descriptor <...>, like this one:

[1/4/2023, 2:04:23 PM] ERROR Error-Handler: SyncError (FATAL): Sync failed caused by LibraryError (FATAL): Error while converting asset caused by LibraryError (FATAL): Unknown filetype descriptor public.mpeg (with potential extension .mpg)

This means that you have an asset in your library whose filetype is not yet supported.

Please report this filetype descriptor in this thread to include it in future releases. In the meantime you can add the filetype descriptor as key and the file extension as value in the above mapped file of your local icloud-photos-sync installation.

In the above example, the required key/value pair would be 'public.mpeg: 'mpg'.

@steilerDev steilerDev added the class(known issue) A known issue that might not be easy to fix or needs additional input. label Jan 4, 2023
@Laffreux-dev
Copy link

Laffreux-dev commented Jan 8, 2023

Hello,

Thank you for this promising app !

Experienced fatal error at 1/8/2023, 4:38:23 PM: SyncError (FATAL): Sync failed caused by LibraryError (FATAL): Error while converting asset caused by LibraryError (FATAL): Unknown filetype descriptor public.avi (with potential extension .avi) (Error Code: 39997bc9-7aed-4162-b2ff-244658dee643)

I tried 'com.microsoft.avi': 'avi', in file-type.ts without success :(

i'm stuck and don't know how to find and remove this file from icloud.

Any idea ?

Thanks,

@steilerDev
Copy link
Owner Author

Try 'public.avi': 'avi' in the file-type.ts, this should fix it :)

I'll include 'avi' in the next one :)

@Laffreux-dev
Copy link

Laffreux-dev commented Jan 8, 2023

ok the filetype descriptor was in the error message🤦‍♂️
It's ok now, but now I get a new error

Experienced fatal error at 1/8/2023, 4:51:00 PM: SyncError (FATAL): Sync failed caused by iCloudError (FATAL): Unable to fetch records for album 'All photos' caused by Request failed with status code 500 (Error Code: f6cf423a-61d6-4682-8f0d-75c9bb463f3a)

@Laffreux-dev
Copy link

I have the impression that the error on the status code 500 is temporary, I come across the unknow error again

Experienced fatal error at 1/8/2023, 5:05:12 PM: SyncError (FATAL): Sync failed caused by LibraryError (FATAL): Error while converting asset caused by LibraryError (FATAL): Unknown filetype descriptor public.avi (with potential extension .avi) (Error Code: 1aefd397-dc4b-42f0-be2e-0f7586d85bd8)

I modified the file-type.ts file to add 'public.avi': 'avi', then relaunched the container, is that enough?

@steilerDev
Copy link
Owner Author

steilerDev commented Jan 8, 2023

You'd need to modify the file located at /opt/icloud-photos-sync/bin/lib/photos-library/model/file-type.js within the container. A relaunch should be sufficient, but I'm pretty sure that the change will not persist between runs, since you are modifying parts of the container and not a volume (this is the area, where I'm not 100% sure how Docker behaves :D )

Could you try installing it through npm directly and modifying it in your local install dir?

@Laffreux-dev
Copy link

You're right, I was modifying the file locally, not the one in the container, I mounted ./app on /opt/icloud-photos-sync and it's ok now.

Work is in progress
image

Thank you !

@steilerDev
Copy link
Owner Author

@Laffreux-dev nightly Docker image contains change for avi support

@Laffreux-dev
Copy link

Great, thank you !

@steilerDev steilerDev pinned this issue Jan 9, 2023
@steilerDev steilerDev changed the title LibraryError (FATAL): Unknown filetype descriptor LIBRARY_UNKNOWN_FILE_TYPE_DESCRIPTOR (FATAL): Unknown filetype descriptor Mar 13, 2023
@jaco
Copy link

jaco commented Apr 24, 2023

Hello, I've got :

Experienced fatal error at 4/24/2023, 1:12:49 PM: APP_SYNC (FATAL): Sync failed caused by SYNC_CONVERSION (FATAL): Error while converting asset caused by LIBRARY_UNKNOWN_FILE_TYPE_DESCRIPTOR (FATAL): Unknown filetype descriptor, see GH issue 143 (dyn.ah62d4rv4ge8022pwq7w1k4dzrzvg62pmru (with potential extension .largeThumbnail))

Is there any way to skip those files?

@orofino
Copy link

orofino commented May 3, 2023

Just received the error for com.olympus.or-raw-image (with potential extension .ORF)

EDIT: As in the above, adding this to file-type.ts fixed the issue.

@ionothanus
Copy link

Just received
Unknown filetype descriptor, see GH issue 143 (com.canon.cr3-raw-image (with potential extension .CR3)) (Error Code: b19e770a-dcae-4f19-898d-9d91a9a14a8e)

fixed by adding

'com.canon.cr3-raw-image': `cr3`,

to file-types.js as described.

@txwireless
Copy link

txwireless commented Jun 14, 2023

I had to also add these two:

'com.adobe.illustrator.ai-image': `ai`,
'public.avchd-mpeg-2-transport-stream': `m2ts`,

I did find a potential source for a more comprehensive list:
https://gist.github.com/rmorey/b8d1b848086bdce026a9f57732a3b858

I tried parsing the output of the frameworks library differently and found that tags: holds the corresponding file extension but it is a messy dump file with other things in it that wouldn't make sense. Maybe if I have some more time I'll try to figure out how to generate a comprehensive list.

It would also be nice if this tool didn't abort, but just skipped any files that didn't match a known one.

@steilerDev steilerDev added class(bug) Something isn't working status(help needed) Help from other people is necessary to resolve this issue and removed class(bug) Something isn't working labels Jul 5, 2023
@Sowevo
Copy link

Sowevo commented Jul 7, 2023

hello
please add
'com.dji.mimo.pano.jpeg' : jpg

@lonevvolf
Copy link

Here's another:
'public.mpo-image': MPO

@steilerDev
Copy link
Owner Author

steilerDev commented Jul 9, 2023

MPO and jpg (and all others until so far) added in v.1.1.1-nightly.14

@teh-hippo
Copy link

Hi!

Please add AVIF:
public.avif: 'avid'

Experienced fatal error at 8/4/2023, 11:43:03 AM: APP_SYNC (FATAL): Sync failed caused by SYNC_CONVERSION (FATAL): Error while converting asset caused by LIBRARY_UNKNOWN_FILE_TYPE_DESCRIPTOR (FATAL): Unknown filetype descriptor, see GH issue 143 (public.avif (with potential extension .AVIF)) (Error Code: 2be417bf-d969-419c-a7c7-c17b646159be)

@steilerDev
Copy link
Owner Author

Added to 1.2.0-nightly.6

@steilerDev
Copy link
Owner Author

1.2.0-nightly.7 no longer fatally fails when there is an unknown file type descriptor, but ignores the asset without file type support. If --enable-crash-reporting is enabled the extension will be reported using the crash handler.

@bassrock
Copy link

bassrock commented Aug 14, 2023

I use Unraid to store/backup too and got an issue with unraids hidden .fuse* files.

LIBRARY_INVALID_FILE (WARN): Found invalid file (.fuse_hidden0016736000000ccc) caused by LIBRARY_UNKNOWN_FILETYPE_EXTENSION (FATAL): Unknown filetype extension, see GH issue 143 ()

@steilerDev
Copy link
Owner Author

Added files matching /^.fuse_hidden.*$/ to ignore list with 1.2.0-nightly.11, however this seems to have broken my E2E tests - need to take a look tomorrow/later this week what's happening (so proceed with caution).

@mzaferyahsi
Copy link

Hi, I have just received the following error for Canon Raw Images (CR3)

Experienced fatal error at 8/25/2023, 4:03:07 PM: APP_SYNC (FATAL): Sync failed caused by SYNC_CONVERSION (FATAL): Error while converting asset caused by LIBRARY_UNKNOWN_FILE_TYPE_DESCRIPTOR (FATAL): Unknown filetype descriptor, see GH issue 143 (com.canon.cr3-raw-image (with potential extension .CR3)) (Error Code: d4469981-c522-4308-ba5c-04feab5bb97a)

@steilerDev
Copy link
Owner Author

@mzaferyahsi this was already previously reported and added in v.1.1.1-nightly.14 - so going with the latest beta should resolve this.

@vguerci
Copy link

vguerci commented Sep 2, 2023

Hey, just had to add 'com.adobe.pdf': `.pdf`,

What about to report these as WARNING instead of Fatal errors? I don't care much if one pdf is missing amongst thousands of photos. Or allow to ignore these with an option.

Thank you 🙏

@steilerDev
Copy link
Owner Author

@vguerci making this non-fatal has already been implemented in the latest beta :)

@mzaferyahsi
Copy link

Hey, I've also received the following warnings;

[2023-09-11T14:12:57.626Z] WARN RuntimeWarning: Unknown file extension .HEIC for descriptor public.image
[2023-09-11T14:12:57.626Z] WARN RuntimeWarning: Error while loading iCloud asset 60AA40EE-A450-4530-BE69-B284CA6B35E4: LIBRARY_UNKNOWN_FILETYPE_DESCRIPTOR: Unknown filetype descriptor, please report in GH issue 143 (public.image (with potential extension .HEIC))
[2023-09-11T14:12:57.627Z] WARN RuntimeWarning: Unknown file extension .HEIC for descriptor public.image
[2023-09-11T14:12:57.627Z] WARN RuntimeWarning: Error while loading iCloud asset 7C23AF0A-19E8-4937-A498-4C82C298B3E4: LIBRARY_UNKNOWN_FILETYPE_DESCRIPTOR: Unknown filetype descriptor, please report in GH issue 143 (public.image (with potential extension .HEIC))
[2023-09-11T14:12:57.628Z] WARN RuntimeWarning: Unknown file extension .HEIC for descriptor public.image
[2023-09-11T14:12:57.628Z] WARN RuntimeWarning: Error while loading iCloud asset 52C23F77-2196-4DE3-B5B0-50CD28E59328: LIBRARY_UNKNOWN_FILETYPE_DESCRIPTOR: Unknown filetype descriptor, please report in GH issue 143 (public.image (with potential extension .HEIC))
[2023-09-11T14:12:57.629Z] WARN RuntimeWarning: Unknown file extension .HEIC for descriptor public.image
[2023-09-11T14:12:57.629Z] WARN RuntimeWarning: Error while loading iCloud asset ED0DF5BB-791D-456A-8E82-25F54598E0CC: LIBRARY_UNKNOWN_FILETYPE_DESCRIPTOR: Unknown filetype descriptor, please report in GH issue 143 (public.image (with potential extension .HEIC))
[2023-09-11T14:12:57.630Z] WARN RuntimeWarning: Unknown file extension .HEIC for descriptor public.image
[2023-09-11T14:12:57.630Z] WARN RuntimeWarning: Error while loading iCloud asset BE472A0F-3E22-4FD4-B38A-2E2B83898DB8: LIBRARY_UNKNOWN_FILETYPE_DESCRIPTOR: Unknown filetype descriptor, please report in GH issue 143 (public.image (with potential extension .HEIC))
[2023-09-11T14:12:57.630Z] WARN RuntimeWarning: Unknown file extension .HEIC for descriptor public.image
[2023-09-11T14:12:57.630Z] WARN RuntimeWarning: Error while loading iCloud asset C64F9506-E473-4438-A4F9-0C5D8CDC9D24: LIBRARY_UNKNOWN_FILETYPE_DESCRIPTOR: Unknown filetype descriptor, please report in GH issue 143 (public.image (with potential extension .HEIC))
[2023-09-11T14:12:57.631Z] WARN RuntimeWarning: Unknown file extension .HEIC for descriptor public.image
[2023-09-11T14:12:57.631Z] WARN RuntimeWarning: Error while loading iCloud asset EB6B9522-10B9-416B-8FBD-652EEC023F0A: LIBRARY_UNKNOWN_FILETYPE_DESCRIPTOR: Unknown filetype descriptor, please report in GH issue 143 (public.image (with potential extension .HEIC))
[2023-09-11T14:12:57.632Z] WARN RuntimeWarning: Unknown file extension .HEIC for descriptor public.image
[2023-09-11T14:12:57.632Z] WARN RuntimeWarning: Error while loading iCloud asset 3E650C0A-6344-4E69-A181-97EFE1AE0A20: LIBRARY_UNKNOWN_FILETYPE_DESCRIPTOR: Unknown filetype descriptor, please report in GH issue 143 (public.image (with potential extension .HEIC))
[2023-09-11T14:12:57.633Z] WARN RuntimeWarning: Unknown file extension .HEIC for descriptor public.image
[2023-09-11T14:12:57.633Z] WARN RuntimeWarning: Error while loading iCloud asset E0FAFBE7-168C-448A-B04A-5C521C7141A2: LIBRARY_UNKNOWN_FILETYPE_DESCRIPTOR: Unknown filetype descriptor, please report in GH issue 143 (public.image (with potential extension .HEIC))

@steilerDev steilerDev changed the title LIBRARY_UNKNOWN_FILE_TYPE_DESCRIPTOR (FATAL): Unknown filetype descriptor "Warning: Detected unknown filetype (descriptor with ext)" Sep 18, 2023
@steilerDev
Copy link
Owner Author

public.image/HEIC will be included in the next release - interessting enough though, the lower case heic extension pops up as image.heic

@lonevvolf
Copy link

Got the following with 1.2.0:
Warning: Detected unknown filetype (undefined with MPO): Please report in GH issue 143

@steilerDev
Copy link
Owner Author

undefined is problematic - seems I actually need to get rid of filetype descriptors...

@d0ugal
Copy link

d0ugal commented Oct 14, 2024

I got this on 1.3.0

Warning: Detected unknown filetype (com.microsoft.waveform-audio with .wav): Please report in GH issue 143

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
class(known issue) A known issue that might not be easy to fix or needs additional input. status(help needed) Help from other people is necessary to resolve this issue
Projects
None yet
Development

No branches or pull requests