Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Features/migration cyberduck release windows chocolatey #15479

Draft
wants to merge 45 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
490a3c2
set distributionManagement repos to local
RaveenaRamany Dec 10, 2023
ccabc97
Merge pull request #1 from RaveenaRamany/features/migration-cyberduck…
RaveenaRamany Dec 10, 2023
7c6015a
Add workflow cyberduck-release-windows
RaveenaRamany Dec 10, 2023
8795297
Fix missing transforms
RaveenaRamany Dec 10, 2023
5bdd666
update Repository secrets
RaveenaRamany Dec 10, 2023
70e22c7
Fix secret names
RaveenaRamany Dec 10, 2023
b05f3ef
Change runner to github hosted windows runner
RaveenaRamany Dec 10, 2023
c7ad044
Fix typo in secret name
RaveenaRamany Dec 10, 2023
e9297de
Merge pull request #2 from RaveenaRamany/convert-cyberduck-release-wi…
RaveenaRamany Dec 10, 2023
f67f6ef
Add option for git tag
RaveenaRamany Dec 10, 2023
3404b02
Merge pull request #3 from RaveenaRamany/features/migration-cyberduck…
RaveenaRamany Dec 10, 2023
547c0b8
Fix WORKSPACE variable
RaveenaRamany Dec 10, 2023
9f7cd51
Merge pull request #4 from RaveenaRamany/features/migration-cyberduck…
RaveenaRamany Dec 10, 2023
52633d0
Add workflow cyberduck-release-windows-chocolatey
RaveenaRamany Dec 11, 2023
b415011
Merge pull request #5 from RaveenaRamany/convert-cyberduck-release-wi…
RaveenaRamany Dec 11, 2023
dc9cbb6
comment env-key
RaveenaRamany Dec 11, 2023
4750959
Update cyberduck-release-windows-chocolatey.yml
RaveenaRamany Dec 12, 2023
06ca975
fix Variable GITHUB_REPOSITROY
RaveenaRamany Dec 14, 2023
40c20d8
update runs-on
RaveenaRamany Dec 14, 2023
815aa50
rewrite in Powershell
RaveenaRamany Dec 14, 2023
0ad8271
change file destination for test
RaveenaRamany Dec 14, 2023
b380e2c
testing nupkg file
RaveenaRamany Dec 14, 2023
00d80a2
create sourcefolder
RaveenaRamany Dec 14, 2023
18fed8d
fix space error
RaveenaRamany Dec 14, 2023
f2accee
fix source path
RaveenaRamany Dec 14, 2023
26c5609
add file to sourcepath
RaveenaRamany Dec 14, 2023
2d2803e
choco push test
RaveenaRamany Dec 14, 2023
ed95adf
choco push test
RaveenaRamany Dec 14, 2023
2aeb009
remove debug
RaveenaRamany Dec 14, 2023
1b3a174
test with cyberduck choco package
RaveenaRamany Dec 14, 2023
614ca1c
local file choco push test
RaveenaRamany Dec 14, 2023
a8900b9
enable debugging
RaveenaRamany Dec 14, 2023
8f0dbec
fix path
RaveenaRamany Dec 14, 2023
99d5054
debug path
RaveenaRamany Dec 14, 2023
56851c4
choco push repo test
RaveenaRamany Dec 14, 2023
c8a712e
set default push source
RaveenaRamany Dec 14, 2023
b8776ac
configure choco api key
RaveenaRamany Dec 14, 2023
1011f7d
fix powershell env syntax
RaveenaRamany Dec 14, 2023
d095c52
Add git nupkg for testing
RaveenaRamany Dec 14, 2023
3f8e370
add local choco repo
RaveenaRamany Dec 14, 2023
40441fb
change local path
RaveenaRamany Dec 14, 2023
dc162ea
add api key
RaveenaRamany Dec 14, 2023
3496a30
remove old batch command
RaveenaRamany Dec 14, 2023
8ba0f95
Add comments
RaveenaRamany Dec 17, 2023
686884d
Add comments
RaveenaRamany Dec 17, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions .github/workflows/cyberduck-release-windows-chocolatey.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
name: cyberduck-release-windows-chocolatey
on:
workflow_dispatch:
env:
CHOCOLATEY_API: "${{ secrets.CHOCOLATEY_API_KEY }}"
jobs:
build:
runs-on:
- windows-latest
steps:
- name: checkout
uses: actions/[email protected]
- name: Publish to Chocolatey
shell: powershell
run: choco config set --name="'defaultPushSource'" --value="${Env:GITHUB_WORKSPACE}\..\"; choco apikey add -k ${Env:CHOCOLATEY_API} -s "${Env:GITHUB_WORKSPACE}\..\"; Get-ChildItem "${Env:GITHUB_WORKSPACE}\0-TEMP-CHOCO\" -Filter *.nupkg | Foreach-Object -process { choco push $_.FullName --debug }
# TODO: The following line must be used for production
# choco config set --name="'defaultPushSource'" --value="'https://push.chocolatey.org/'"; choco apikey add -k ${Env:CHOCOLATEY_API} -s https://push.chocolatey.org/; Get-ChildItem "${Env:GITHUB_WORKSPACE}\..\cyberduck-release-windows\windows\target\release\" -Filter cyberduck.*.nupkg | Foreach-Object -process { choco push $_.FullName --debug }

# TODO: Github only allows sending notifications to the actor of the GitHub Action: https://docs.github.com/en/actions/monitoring-and-troubleshooting-workflows/notifications-for-workflow-runs
# Mailer plugin was not converted because GitHub Actions will email the actor after failed build and does not support emailing a list of recipients
161 changes: 161 additions & 0 deletions .github/workflows/cyberduck-release-windows.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
name: cyberduck-release-windows
on:
workflow_dispatch:
inputs:
GIT_TAG:
required: false
env:
GITHUB_REPOSITORY: "${{ github.workspace }}\\${{ github.event.repository.name }}"
AWS_ACCESS_KEY_DEPLOYMENT: "${{ secrets.AWS_ACCESS_KEY_DEPLOYMENT }}"
AWS_SECRET_KEY_DEPLOYMENT: "${{ secrets.AWS_SECRET_KEY_DEPLOYMENT }}"
RACKSPACE_USER: "${{ secrets.RACKSPACE_USER }}"
RACKSPACE_PASSWORD: "${{ secrets.RACKSPACE_PASSWORD }}"

jobs:
build:
runs-on:
- windows-latest
steps:
- name: clean workspace
shell: ruby {0}
run: |-
require "fileutils"
Dir.chdir(ENV["GITHUB_WORKSPACE"]) do
paths = Dir.glob(["**/*"])
paths -= Dir.glob([".git/**", ".repository/**"])
paths.each do |path|
File.delete(path) if File.file?(path)
FileUtils.rm_rf(path) if File.directory?(path)
end
end
- name: checkout
uses: actions/[email protected]
with:
ref: ${{ inputs.GIT_TAG }}
- name: Copy Sparkle Updater Private Key (DSA)
shell: powershell
env:
SPARKLE_UPDATER_PK: ${{ secrets.WIN_SPARKLE_UPDATER_PRIVATE_KEY }}
run: ${env:SPARKLE_UPDATER_PK} > $GITHUB_WORKSPACE\www\update\private.pem

- name: Set up JDK 17
uses: actions/[email protected]
with:
distribution: 'temurin'
java-version: '17'
settings-path: "${{ github.workspace }}"
- name: Generate maven settings.xml
uses: s4u/[email protected]
with:
servers: |
[{
"id": "maven.iterate.ch-release",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maven.iterate repositories not necessary for Cyberduck builds.

"username": "AKIAI44PZB4OH5YGWFVA",
"password": "${{ secrets.MAVEN_SERVER_PASSWORD }}",
},
{
"id": "maven.iterate.ch-snapshot",
"username": "AKIAI44PZB4OH5YGWFVA",
"password": "${{ secrets.MAVEN_SERVER_PASSWORD }}",
},
{
"id": "maven.cyberduck.io-release",
"username": "AKIAI44PZB4OH5YGWFVA",
"password": "${{ secrets.MAVEN_SERVER_PASSWORD }}",
},
{
"id": "maven.cyberduck.io-snapshot",
"username": "AKIAI44PZB4OH5YGWFVA",
"password": "${{ secrets.MAVEN_SERVER_PASSWORD }}",
}]
- name: Run maven
run: mvn clean deploy --settings $GITHUB_REPOSITORY/settings.xml -DskipTests -D"sparkle.feed="
- name: run batch command
shell: cmd
run: FOR %%c in ("%GITHUB_REPOSITORY%\windows\target\release\Cyberduck-*") DO "C:\Program Files\Cyberduck CLI\duck.exe" --upload rackspace:/cdn.cyberduck.ch/ "%%c" --username %RACKSPACE_USER% --password %RACKSPACE_PASSWORD% --retry --quiet --existing overwrite --region DFW
- name: run batch command
shell: cmd
run: FOR %%c in ("%GITHUB_REPOSITORY%\windows\target\release\Cyberduck-*") DO "C:\Program Files\Cyberduck CLI\duck.exe" --upload s3:/release.cyberduck.io/ "%%c" --username %AWS_ACCESS_KEY_DEPLOYMENT% --password %AWS_SECRET_KEY_DEPLOYMENT% --retry --quiet --existing overwrite --region us-east-1
- name: Upload Artifacts
uses: actions/[email protected]
if: always()
with:
path: |-
windows\target\release\*.exe
windows\target\release\*.msi
windows\target\release\*.appx
!**/*~
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need all these exclusions? Are the 3 includes above not sufficient?

!**/#*#
!**/.#*
!**/%*%
!**/._*
!**/CVS
!**/CVS/**
!**/.cvsignore
!**/SCCS
!**/SCCS/**
!**/vssver.scc
!**/.svn
!**/.svn/**
!**/.DS_Store
!**/.git
!**/.git/**
!**/.gitattributes
!**/.gitignore
!**/.gitmodules
!**/.hg
!**/.hg/**
!**/.hgignore
!**/.hgsub
!**/.hgsubstate
!**/.hgtags
!**/.bzr
!**/.bzr/**
!**/.bzrignore
# # Fingerprinter was not converted because the behavior is available by default in GitHub Actions and/or it is not configurable
# # Mailer plugin was not converted because GitHub Actions will email the actor after failed build and does not support emailing a list of recipients
# Ensure parameter if_key_exists is set correctly
- name: Install SSH key
uses: shimataro/[email protected]
with:
key: "${{ secrets.VERSION_CYBERDUCK_IO_SSH_KEY }}"
name: id_rsa-version_cyberduck_io
known_hosts: "${{ secrets.VERSION_CYBERDUCK_IO_KNOWN_HOSTS }}"
if_key_exists: replace # replace will allow us to update the ssh-key on an existing build
config: |
Host VERSION_CYBERDUCK_IO
HostName ${{ secrets.VERSION_CYBERDUCK_IO_HOST_NAME }}
User ${{ secrets.VERSION_CYBERDUCK_IO_USER }}
IdentityFile ~/.ssh/id_rsa-version_cyberduck_io
if: always()
- name: setup file transfer file
uses: actions/[email protected]
with:
script: |-
const fs = require('fs').promises
const path = require('path')
const patterns = "windows/target/update/changelog.*,!**/*~,!**/#*#,!**/.#*,!**/%*%,!**/._*,!**/CVS,!**/CVS/**,!**/.cvsignore,!**/SCCS,!**/SCCS/**,!**/vssver.scc,!**/.svn,!**/.svn/**,!**/.DS_Store,!**/.git,!**/.git/**,!**/.gitattributes,!**/.gitignore,!**/.gitmodules,!**/.hg,!**/.hg/**,!**/.hgignore,!**/.hgsub,!**/.hgsubstate,!**/.hgtags,!**/.bzr,!**/.bzr/**,!**/.bzrignore"
Copy link
Contributor

@ylangisc ylangisc Dec 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question here with all the excludes. If not necessary the entire transfer task and ssh setup could be simplified using https://github.com/appleboy/scp-action.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice if we can dogfood Cyberduck CLI here. We have a Cyberduck CLI Github Action 1 to transfer artifacts. Documentation can be found in the README 2 and sample usage in our connection profiles synchronization with S3 3.

@AliveDevil May assist here when in doubt.

Footnotes

  1. https://github.com/iterate-ch/cyberduck-cli-action

  2. https://github.com/iterate-ch/cyberduck-cli-action/blob/main/README.md

  3. https://github.com/iterate-ch/profiles/blob/main/.github/workflows/profiles-sync.yml

const globber = await glob.create(patterns.replace(/,/g, "\n"))
const files = []
for await (const file of globber.globGenerator()) {
if ((await fs.lstat(file)).isDirectory()) continue
files.push(path.relative(process.cwd(), file))
}
fs.writeFile("version_cyberduck_io-transfer.txt", files.join("\n"), (err) => {})
if: always()
- name: run file transfers
run: |-
ssh VERSION_CYBERDUCK_IO 'mkdir -p windows'
tar -cvf version_cyberduck_io-transfer.tar --files-from version_cyberduck_io-transfer.txt
scp version_cyberduck_io-transfer.tar VERSION_CYBERDUCK_IO:
ssh VERSION_CYBERDUCK_IO 'tar -xvf version_cyberduck_io-transfer.tar -C windows && rm version_cyberduck_io-transfer.tar'
if: always()
# # This item has no matching transformer
# - uses: hudson.tasks.BuildTrigger
# with:
# childProjects: cyberduck-release-windows-chocolatey
# threshold:
# name: SUCCESS
# ordinal: '0'
# color: BLUE
# completeBuild: 'true'
Binary file added 0-TEMP-CHOCO/git.2.43.0.nupkg
Binary file not shown.
6 changes: 4 additions & 2 deletions pom.xml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs to be reverted before merge.

Original file line number Diff line number Diff line change
Expand Up @@ -869,12 +869,14 @@
<distributionManagement>
<repository>
<id>maven.cyberduck.io-release</id>
<url>s3://repo.maven.cyberduck.io/releases/</url>
<!-- <url>s3://repo.maven.cyberduck.io/releases/</url>
-->
<url>file://${user.home}</url>
<layout>default</layout>
</repository>
<snapshotRepository>
<id>maven.cyberduck.io-snapshot</id>
<url>s3://repo.maven.cyberduck.io/snapshots/</url>
<url>file://${user.home}</url>
</snapshotRepository>
<site>
<id>github-pages-site</id>
Expand Down