Add support for uploading multiple artifacts

[aguibert]

Closes #125

Summary:

Adds a new optional option to upload artifacts individually instead of all in one. Default behavior remains unchanged. To specify the new behavior, the config looks like:

    - name: Upload individual zips
      uses: actions/upload-artifact@vWhatever
      with:
        name: dummy
        path: dummy/*.zip
        single-archive: "false"

Motivation:

In my github actions pipeline we run a series of test projects in matrix jobs and then zip up relevant logs. Each matrix jobs runs 3-7 test projects. We would like to upload these logs only failures are detected in each project (so 3-7 zips per matrix job). Currently the 3-7 zips get added to a larger zip and then uploaded, but we would like to keep them separate so we don't need to download the larger zip to check for the project's logs we care about.

Disclaimer: I'm a Java developer and have never worked with Typescript before so I did my best to code this up properly.

[jvbsl]

Any progress on this? I would be quite interested to upload multiple artifacts.
The "failed" tests did not fail for me on macos-latest and ubuntu-latest.
If not is there any possibility to upload multiple artifacts in some other way?

[yacaovsnc]

hi @jvbsl It's already possible to upload multiple artifacts in a single workflow - you just need to define multiple steps of this action and each step can use a unique name, thus creating multiple artifacts.

I wonder if you could make this to work steps context . This maybe a bit cumbersome, but if you can break your tests in separate steps, and then create an artifact only if the test fails?

I am going to close this PR since the suggested syntax could be misused, it's easy to create many artifacts accidentally and that may surprise people and create some scalability issue. This requires some design up front before we code the solution.

[jvbsl]

I don't know the artifact names or count beforehand so if it is not possible to dynamically create steps I don't see how I could do that(and it would be a huge workaroun for smth so simple).

Additionaly I don't see how one could "accidentally" create multiple artifacts with the whole thing being controlled by an imho pretty clear option(with that argument we should remove the name option as well, because users could accidentally upload an artifact using an incorrect name^^)

Would be great if you could reopen the PR and instead say what it would need for you to agree to it.
Thanks ;)

[yacaovsnc]

Sure, this is open for discussion. :)

I don't see how one could "accidentally" create multiple artifacts

With this syntax: path: dummy/*.zip, I assume we are going to create one artifact per zip file? Since this is a simple glob, there is nothing to stop anyone to write *.js, or *.java, and in that case, we will create many unwanted artifacts.

This is really not about the option not been clear, but there is no safe guard against number of artifacts we can create in one simple step.

Given the scenario, I wonder if it will be better solved by the ability to list and download files in a single artifact selectively. For example, if there are 7 zips in this artifact, we can list and download the only one we need instead of all 7.

[AraHaan]

I would like to use the action to cache the results from dotnet restore and dotnet build for my other jobs in my projects default branch and pull request workflows (which ironically depends on the build job passing), then it will do the security scan job, itself I want it to use the files from the build job in the workflow so I do not have to execute dotnet restore or dotnet build again, then when that pass run dotnet test in a test job, then when that pass run benchmarkdotnet to benchmark the codebase in a bench job...

As such I would need some way to cache the information from the build job (it takes a few minutes itself to fully build (4~6 minutes) so the need to repeat those over and over does not sound great to me. Other than using this action and then the download-artifact action in the other jobs.

Bonus points: if I can make a job for after those jobs done that can delete the artifacts triggered from that workflow run as well since they should not be used for other CI invocations for the sake of things can happen and the c# compiler emitting stale dlls instead of recompiling them from scratch every time.

Even greater bonus points: Forcing actions/checkout to not clear out every file from the previous job. It seems to me it tries to sync it but then later delete it all and clone it all over again. I could try to see what will happen if I remove actions/checkout from those other jobs.

[jvbsl]

@yacaovsnc Thanks for reopening and helping me understand what you meant.

Given the scenario, I wonder if it will be better solved by the ability to list and download files in a single artifact selectively. For example, if there are 7 zips in this artifact, we can list and download the only one we need instead of all 7.

Well for me it would be more of a thing that a User can choose the specific zip on the release page, if it would be possible to choose from there that would be fine too, but I don't think that was what you meant?

Too prevent someone from accidentally uploading wrong files, how about something like this instead:

     with:
        name: dummy
        path: dummy/*.zip
        single-archive-types: [ "*.tar.*", "*.zip" ]

If someone at that point uses something like single-archive-types: [ "*" ] it is their own fault and would be an extra layer of accidental misuse. Would that be a solution you could go with?

[yacaovsnc]

I would like to use the action to cache the results from dotnet restore and dotnet build for my other jobs in my projects default branch and pull request workflows

@AraHaan have you looked at the cache action https://github.com/actions/cache yet? It sounds like a better fit for what you are trying to achieve.

[yacaovsnc]

Well for me it would be more of a thing that a User can choose the specific zip on the release page, if it would be possible to choose from there that would be fine too, but I don't think that was what you meant?

@jvbsl yes I meant from the UI user will have a way to select which files to download inside the artifact. This is something we have been thinking about, but didn't really make any progress yet.

it is their own fault and would be an extra layer of accidental misuse.

That is correct, however we do have to consider the abuse scenario, this will be a DoS attack vector on the backend server.

[JasonGross]

Since this is a simple glob, there is nothing to stop anyone to write *.js, or *.java, and in that case, we will create many unwanted artifacts.

Maybe you can solve this by adding an optional argument "max-separate-archives" which defaults to 10 or something like that?

[ArafatMohammedSalehAlialnmari]

جوجل

[initdc]

#354

[hiddify-com]

it still create .zip for each indiviusal file?