I don't understand the rate limiting... and bulk operations

[joshiste]

... What I basically need to to is to delete a huge amount (>5000) of tags on a private repo of our paid org.
Wrote a little nice bash script using hub-tool for this purpose, as deleting it via advanced image management is a PITA. Now my problem:

After ~820 deletes I got an "429 too many requests".

querying the rate limiting gives:

hub-tool account rate-limiting
Limit:     100, 6 hours window
Remaining: 100, 6 hours window

But even getting my account info fails:

hub-tool account info
Error: failed to authenticate: bad status code "429 Too Many Requests": Docker Hub API rate limit exceeded, contact [email protected] to give us feedback

So I guess I'm facing two issues here:

1. a bulk operation to delete multiple tags at once
2. good information (or event better gracefully handling) of rate limits.

[ingshtrom]

It sounds like you are hitting our overall rate limits. These are in place for all customers to protect the Docker Hub services and the solution will be to "slow down".

My recommendation would be to check the exit code for failure and slow down. Something as simple as if [[ hitRateLimit -eq 'true' ]]; then sleep 30; fi would be a good addition to your script. You could also add a smaller sleep between tag deletions, depending on how your script works, so that you avoid hitting the limit altogether 😄