Upload Error - Invalid Argument - 0.23.0-alpha5 - Windows

[hellocharli]

Just hit my Google Photos free storage limit and I'm super excited to switch over to Immich. Unfortunately I'm getting this error on the first image upload. Dry runs go through perfectly. Running it with --sync-albums=false gets rid of everything after the ERR line, but I think that's just from the first error then creating a bad POST request... I could be wrong though.

2024-11-14 00:07:04 INF associated metadata file file=takeout-20241114T050723Z-001:Takeout/Google Photos/album1/IMG_9953.JPG json=IMG_9953.JPG.json matcher=normalMatch
2024-11-14 00:07:04 ERR upload error file=takeout-20241114T050723Z-001:Takeout\Google Photos\album2\IMG_1267.HEIC error=open Takeout\Google Photos\album2\IMG_1267.HEIC: invalid argument
2024-11-14 00:07:04 INF error file= !BADKEY=CreateAlbum, POST, http://10.0.0.10:2283/api/albums, 400 Bad Request
Bad Request
each value in assetIds must be a UUID

2024-11-14 00:07:04 INF added to an album file=takeout-20241114T050723Z-001:Takeout\Google Photos\album2\IMG_1267.HEIC Album=album2
2024-11-14 00:10:53 INF 
2024-11-14 00:10:53 INF Input analysis:
2024-11-14 00:10:53 INF ---------------
2024-11-14 00:10:53 INF scanned image file                      :    8472
2024-11-14 00:10:53 INF scanned video file                      :    1751
2024-11-14 00:10:53 INF scanned sidecar file                    :    9080
2024-11-14 00:10:53 INF discarded file                          :       0
2024-11-14 00:10:53 INF unsupported file                        :       4
2024-11-14 00:10:53 INF file duplicated in the input            :       0
2024-11-14 00:10:53 INF associated metadata file                :   10223
2024-11-14 00:10:53 INF missing associated metadata file        :       0
2024-11-14 00:10:53 INF 
2024-11-14 00:10:53 INF Uploading:
2024-11-14 00:10:53 INF ----------
2024-11-14 00:10:53 INF uploaded                                :       0
2024-11-14 00:10:53 INF upload error                            :       1
2024-11-14 00:10:53 INF file not selected                       :       0
2024-11-14 00:10:53 INF server's asset upgraded with the input  :       0
2024-11-14 00:10:53 INF server has same asset                   :       0
2024-11-14 00:10:53 INF server has a better asset               :       0

And here is the API trace for the failed POST request:

2024-11-14T00:02:12-08:00 QUERY 5 CreateAlbum POST http://10.0.0.10:2283/api/albums
   Accept [application/json]
   Content-Type [application/json]
   X-Api-Key redacted
-- request JSON Body --
{
 "albumName": "album2",
 "description": "",
 "shared": false,
 "assetIds": [
  ""
 ]
}
-- request body end --

Let me know of anything else I can send along or steps to resolve it. Thanks!

[simulot]

Thanks for reporting the error.
I think this happens when the photo exists already on the server, and immich-go tries to add the photo he hasn't uploaded to an album.

[hellocharli]

Got it. Does that mean there's nothing I can do until immich-go gets an update to resolve the issue? Or is there a workaround of some sort?

Edit: I tried to upload my takeout to a fresh docker container and got the same error

Edit again: It worked when using the following command:
.\immich-go.exe upload from-google-photos 'D:\immich-import\takeout\Takeout\Google Photos\Photos from 2024\' --server http://10.0.10.40:2283 --api-key $key --api-trace

It doesn't seem to like parsing the folder path. I had tried to use the takeout zip file too but that gave the same error. I'll try on linux next because I've got nothing better to do

Edit the 3rd: It worked when running this command on linux using the exact same takeout file:
./immich-go upload -google-photos -server http://10.0.0.10:2283 -key $key takeout-20241114T050723Z-001.zip

And then I realized I was using the stable linux release whereas I was running the unstable Windows one. 22.1 on Windows works just fine.

I'll leave it on the stable version for now then. I'll leave this issue open as a bug report for the alpha version specifically, but you can close it if you want.

[simulot]

Edit again: It worked when using the following command:
.\immich-go.exe upload from-google-photos 'D:\immich-import\takeout\Takeout\Google Photos\Photos from 2024' --server http://10.0.10.40:2283 --api-key $key --api-trace
It doesn't seem to like parsing the folder path. I had tried to use the takeout zip file too but that gave the same error. I'll try on linux next because I've got nothing better to do

The path to the archive must be the last argument!

So the invalid argument is happening with the alpha release.

[hellocharli]

Yes, it's happening with the alpha release. I just tried again with this command and it gave me the same errors:
.\immich-go.exe upload from-google-photos -s http://10.0.0.10:2283 -k $key --include-untitled-albums D:\immich-import\takeout-20241114T050723Z-001\
(I moved the import folder to be the last argument and it's still giving the same error)

The command that you just quoted worked on the alpha version but I pointed it to a folder containing only photos and json files directly. It did not work with nested folders or the zipped takeout.

Appreciate all your help with this!

[simulot]

I need more context to understand how the error occurs:

Bad Request

Does your immich server contains some images from the album? if yes, are images in the same resolution / compression in the server compared to the incoming data?

Does the error comes at the second run, when some images are already present?

Or it comes all the times?

[hellocharli]

I've been running each of these with fresh docker containers. I've been running docker compose down -v and deleting the library and postgres folders between each try.

[simulot]

So the problem comes when importing your data set on a fresh install using the alpha5

Could you share more data: https://github.com/simulot/immich-go/blob/main/docs/how-to-send-debug-data.md
For better privacy, you can share it with a discord private message @simulot

[simulot]

It appears that the problem is present only when running the client under windows.

[phreakmonkey]

FWIW I'm getting this same error with a Google Takeout archive with the Linux (amd64) client. I tried rebuilding immich-go from the "next" branch to capture the above fix, but the error persists.

It gets about 17% of the way through the archive, then crashes with:

2024-11-19 11:31:41 INF uploaded file=Google Photos:Archive/PXL_20230118_204905284.MP~2.jpg                                                                                                                                                            
2024-11-19 11:31:41 ERR upload error file=Google Photos:Archive/PXL_20230118_204905284.MP~2 error=AssetUpload, POST, http://phoenix:2283/api/assets, 400 Bad Request                                                                                   │
Bad Request                                                                                                                                                                                                                                            

2024-11-19 11:31:41 INF Stacked file=Google Photos:Archive/PXL_20230118_204905284.MP~2.jpg                                                                                                                                                             
2024-11-19 11:31:41 INF Stacked file=Google Photos:Archive/PXL_20230118_204905284.MP~2                                                                                                                                                                 
2024-11-19 11:31:41 ERR Can't create stack error=createStack, POST, http://phoenix:2283/api/stacks, 400 Bad Request                                                                                                                                    
Bad Request                                                                                                                                                                                                                                            
each value in assetIds must be a UUID

$ immich-go version
immich-go version:dev,  commit:0b05fbbc965a8536d3bc083235d5a22a53cb93b0, date:2024-11-18T16:58:15Z

Let me know if I can provide anything else

[simulot]

yeah, this one resists. I'm tempted to make the error not blocking until I get a better understanding

[phreakmonkey]

Weirdly, I ran it again with -l DEBUG and this time after it got the error it kept going and is now uploading the rest of the collection. Is that expected? That the log severity causes it to continue after a 400 Error? 🤷

EDIT: Oh maybe I wasn't giving it long enough to resume on its own previously? Hrmm.

[phreakmonkey]

Okay, I found a commonality between all the files that are returning the error=AssetUpload, POST, http://phoenix:2283/api/assets, 400 Bad Request error. They're all .MP~2 files that have been annotated / edited on my pixel. (Usually by drawing a highlight or circle on the image.)

I had about a dozen such files causing the crash. Every one of them, when viewed, was a stacked (MP4) video that had a highlight or annotation drawn on it.

I've attached one of the culprits that reproduces the error in case that helps chase it down. (gzipped so that githib would let me attach it)
PXL_20220225_235718567.MP~2.gz

[simulot]

Weirdly, I ran it again with -l DEBUG and this time after it got the error it kept going and is now uploading the rest of the collection. Is that expected? That the log severity causes it to continue after a 400 Error? 🤷

Hum, I hope not!

Okay, I found a commonality between all the files that are returning the error=AssetUpload, POST, http://phoenix:2283/api/assets, 400 Bad Request error. They're all .MP~2 files that have been annotated / edited on my pixel. (Usually by drawing a highlight or circle on the image.)

Thank you for your analysis.

[simulot]

BTW the .MP~2 issue isn't fixed by the last commit.
However, work on windows machine problems are solved.

[patdemko]

I think I'm also seeing this issue in the RC version (sample error below with the filename and server url changed). What's confusing to me is if I run it multiple times the file it errors out on isn't always the same one. I would have thought if it was one particular file causing it it should error out at the same one each time I run it if I don't exclude the file.

If you need any other logs on this issue, let me know. Thanks!

2024-11-30 08:11:12 INF Stacked file=takeout-20241128T040914Z-011:Takeout/Google Photos/.../PXL_20210328_150046021.MP~2
2024-11-30 08:11:12 ERR Can't create stack error=createStack, POST, https://url/api/stacks, 400 Bad Request
Bad Request
each value in assetIds must be a UUID

[simulot]

PXL_20210328_150046021.MP~2 The ~2 after the extension may disturb the program.

Is there any reason for stacking this file with another?
You should have other files named PXL_20210328_150046021 around.

Any log would be help full.

[patdemko]

PXL_20210328_150046021.MP~2 The ~2 after the extension may disturb the program.

Is there any reason for stacking this file with another? You should have other files named PXL_20210328_150046021 around.

Any log would be help full.

I'm really not sure sure what the files with the ~ are. I haven't used immich in months until the other day and just did a takeout from google photos (twenty nine, 50GB files) and decided to use the RC version of immich-go try to import them all. That import kept failing on those ~ files until I added a --ban-file '*~*' option to the import to exclude them. I'm still in the process of importing those files, but after I'm done I can try to only import those files and share new logs with you. Is there a debug option you'd like added to the import command to get extra logs?

[simulot]

TLDR: I'm working on the issue, and .MP~2 files are the root cause of the error.

First, MP stands fro Motion Picture (or so). This file contains a short movie sequence taken with the main picture named .MP.JPG files. Both are handled correctly by immich-go / immich stack.

On the pixel phone (at least), you can edit the main photo. The edited photo is now names .MP~2.JPG, witch is perfectly legit, but the movie file is named .MP~2 🤯

The original and the modified files are currently grouped together as a burst of photos. But .MP~2 isn't valid, and the file is rejected by Immich. Then the stack command fails.

I think I'll workaround this issue by rename the couple IMAGE.MP~2.JPG and IMAGE.MP~2 into IMAGE~2.MP.JPG and IMAGE~2.MP