Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: create workflow for cache policy updates #1970

Merged
merged 17 commits into from
Dec 11, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 83 additions & 0 deletions .github/workflows/update-cache-policy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
name: Update cache control policy

on:
workflow_dispatch:
inputs:
policy_type:
type: choice
description: Select the cache control policy type
required: true
options:
- no-store
- max-age=3600

permissions:
id-token: write # allows the JWT to be requested from GitHub's OIDC provider
contents: read # This is required for actions/checkout

jobs:
validate-actor:
runs-on: ubuntu-latest
steps:
- name: Validate if actor is allowed to trigger the workflow
env:
ORG_NAME: rudderlabs
TEAM_NAME: js-sdk
run: |
actor=${{ github.actor || github.triggering_actor }}
response=$(curl -L \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer ${{ secrets.PAT }}" \
-H "X-GitHub-Api-Version: 2022-11-28" \
https://api.github.com/orgs/${{ env.ORG_NAME }}/teams/${{ env.TEAM_NAME }}/memberships/$actor)

if echo "$response" | grep -q '"state": "active"'; then
echo "$actor is a member of $TEAM_NAME team"
else
echo "$actor is NOT a member of $TEAM_NAME team"
exit 1
fi

update-cache-policy:
needs: validate-actor
name: Update cache control policy for SDK artifacts
runs-on: [self-hosted, Linux, X64]

steps:
- name: Install AWS CLI
uses: unfor19/install-aws-cli-action@master

- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::${{ secrets.AWS_PROD_ACCOUNT_ID }}:role/${{ secrets.AWS_PROD_S3_SYNC_ROLE }}
aws-region: us-east-1

- name: Determine the cache control policy
id: determine_policy
run: |
echo "cache_control_policy=${{ github.event.inputs.policy_type || inputs.policy_type }}" >> $GITHUB_ENV

- name: Update cache control policy
run: |
# Get the number of CPU cores in the runner and leave one core free
num_cores=$(nproc --ignore=1 || echo 1) # Default to 1 if nproc is unavailable
# Use a factor to set the parallel jobs (e.g., number of cores or slightly lower)
parallel_jobs=$((num_cores * 2))
echo "Detected $num_cores cores. Using $parallel_jobs parallel jobs."

prefixes=("adobe-analytics-js" "v3" "v1.1")

for prefix in "${prefixes[@]}"; do
echo "Processing prefix: $prefix"

aws s3api list-objects --bucket ${{ secrets.AWS_PROD_S3_BUCKET_NAME }} --prefix "$prefix" --query "Contents[].Key" --output text | tr '\t' '\n' | \
parallel --retries 10 -j "$parallel_jobs" "aws s3api copy-object \
--bucket ${{ secrets.AWS_PROD_S3_BUCKET_NAME }} \
--copy-source ${{ secrets.AWS_PROD_S3_BUCKET_NAME }}/{} \
--key {} \
--metadata-directive REPLACE \
--cache-control '${{ env.cache_control_policy }}'"
done


Loading