Why does imagetools create return 429 with error message regarding pull rate limit?

[ToshY]

I was testing the following command in Github actions (with authenticated dockerhub user):

docker buildx imagetools create -t ***/test:1.2 -t ***/test:1.2.3 ***/test@sha256:1288eceb7fc0fe0f294630126b5112da80410b5ec83e4d6fb4ff87902da4f8fa ***/test@sha256:c5a647b031794a5a639d0afba508d1901cb3d7b1106a7765458db0ccb6f444b3

And at some point I got the following response:

ERROR: httpReadSeeker: failed open: unexpected status code https://registry-1.docker.io/v2/***/test/manifests/sha256:c5a647b031794a5a639d0afba508d1901cb3d7b1106a7765458db0ccb6f444b3: 429 Too Many Requests - 
Server message: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit

As I pointed out that this happened "at some point", it was after dispatching the workflow multiple times, with a matrix strategy, 8 variations (and was also building with bake and matrix strategy in previous steps).

Questions

• Do imagetools create and imagetools inspect impact the "pull" rate limit?
• Is it actually "pulling" the image when performing imagetools create, or is it just a (API) call to the registry to retrieve the manifest?
  • Is "pull" rate limit the correct term to have in this context?
  • Why is there a "pull" rate limit for just fetching a manifest?

The answers to these questions might be obvious for more experienced docker users, but I see no mention in the docker documentation (here or here) that these kind actions also have an impact on the pull rate limit of an authenticated user.

---

Sidenotes

• I'm new to using imagetools commands.
• Example is based on: https://github.com/crazy-max/docker-linguist/blob/64264580e92029d03438d0a86e1acc6c3aa7c139/.github/workflows/build.yml#L185

[crazy-max]

If you're using GHA it should not happen as there is built-in credentials on public runners to pull without rate-limit.

cc @tonistiigi @chris-crone @jcarter3

[ToshY]

If you're using GHA it should not happen as there is built-in credentials on public runners to pull without rate-limit.

Okay, I forgot to mention (maybe a crucial detail) that this issue happened in a private repository (a private clone of ToshY/docker-php). As you've mentiond "public" runners I'm not sure if that means "public" as in difference between GitHub hosted runners and self-hosted runners or if there's a difference between GitHub hosted runners on public or private repositories.

If the latter is the case, I was not aware of this, but that could explain that I've encountered rate limiting.

Even with this explanation, I am still confused on why this causes rate limiting, and that's still the main reason why I am asking. I do not find it very intuitive that a "create" command (like imagetools create) would drain my "pull" rate limit. The error shows a request to the manifest, does this count as "pull"?

[tonistiigi]

The error shows a request to the manifest, does this count as "pull"?

Yes, pulling down a manifest is what impacts the rate limit counter.

[tonistiigi]

https://docs.docker.com/docker-hub/download-rate-limit/?#rate-limit

[ToshY]

https://docs.docker.com/docker-hub/download-rate-limit/?#rate-limit

Thanks for the reply, I totally overlooked that part, my apologies.

Appreciate you all for taking the time to reply 🙂