Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update main to release version 0.3.0 #127

Merged
merged 45 commits into from
Oct 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
ba058c3
Implemented faster-whisper, removed WhisperX
mahenning Jun 18, 2024
f5ef264
add num-speakers as cmdline option to scraibe
Jul 1, 2024
0ddb52c
Merge pull request #107 from tillhanke/main
JSchmie Sep 2, 2024
cf63ac8
update dependencies
JSchmie Sep 9, 2024
533b199
downgraded to pyannote.audio==3.1.1
JSchmie Sep 9, 2024
d25fda5
downgraded to 3.1.1
JSchmie Sep 9, 2024
129f0ce
updated versions
JSchmie Sep 9, 2024
53e57a0
Added tests for faster-whisper
mahenning Sep 9, 2024
5b56b54
Fixed pyannote env var import, now can use `PYANNOTE_CACHE`
mahenning Sep 10, 2024
de9c81b
added language to code support for faster whisper
JSchmie Sep 10, 2024
885d0c8
Fixed pyannote_cache, now looks it up before overwrite.
mahenning Sep 10, 2024
ae1bae7
fixed pypi.yaml to run on push on main
JSchmie Sep 10, 2024
4e7b7e7
Merge pull request #120 from JSchmie/fix_env_var
JSchmie Sep 10, 2024
95c145c
Merge pull request #119 from JSchmie/update-dependencies
mahenning Sep 10, 2024
9ce47ac
Merge pull request #121 from JSchmie/feature_language_support
mahenning Sep 10, 2024
18666ad
Merge pull request #122 from JSchmie/fix-pypi-ci
mahenning Sep 10, 2024
5c0386e
define new Versions of pyannote and faster-whisper
JSchmie Sep 10, 2024
51bf211
updated deps
JSchmie Sep 10, 2024
929f916
Merge branch 'develop' into faster_whisper
JSchmie Sep 10, 2024
ab7b43a
set test whisper model to tiny
JSchmie Sep 10, 2024
df9c510
Merge pull request #123 from JSchmie/faster_whisper
JSchmie Sep 10, 2024
9df0503
align minimal torch version
JSchmie Sep 27, 2024
2adbfae
added relative Path
JSchmie Sep 30, 2024
5f6f681
fix paths
JSchmie Sep 30, 2024
6326d0f
added tests folder
JSchmie Sep 30, 2024
81fb9af
removed old test files
JSchmie Sep 30, 2024
a4b8546
Merge pull request #129 from JSchmie/fix-pytest-action-2
JSchmie Oct 1, 2024
575a8de
Re-fixed PYANNOTE_CACHE, now writes the variable.
mahenning Oct 1, 2024
b07f593
Bump openai-whisper from 20231117 to 20240930
dependabot[bot] Oct 7, 2024
fa1dad6
Update sphinx-rtd-theme requirement from ^2.0.0 to >=2,<4
dependabot[bot] Oct 7, 2024
ce2f3eb
Merge pull request #131 from JSchmie/fix_env_var
JSchmie Oct 8, 2024
6fadf3d
removed torch device from AudioProcessor class
JSchmie Oct 8, 2024
8813662
added SCRAIBE_TORCH_DEVICE to Scraibe Class to handle torch device se…
JSchmie Oct 8, 2024
44ff678
added SCRAIBE_TORCH_DEVICE Variable
JSchmie Oct 8, 2024
af99a65
added SCRAIBE_TORCH_DEVICE to Diariser class
JSchmie Oct 10, 2024
e7c1a5a
added SCRAIBE_TORCH_DEVICE to transcriber class
JSchmie Oct 10, 2024
101e913
make ruff happy
JSchmie Oct 10, 2024
08f1488
Bring dockerfile up to date
mahenning Oct 10, 2024
de90717
Docker image generation workflow from webui
mahenning Oct 10, 2024
f0989a5
Merge pull request #135 from JSchmie/update_dockerfile
JSchmie Oct 10, 2024
3851311
Merge pull request #133 from JSchmie/dependabot/pip/sphinx-rtd-theme-…
JSchmie Oct 11, 2024
fec46aa
Merge branch 'roll-back-torch-verision' into dependabot/pip/openai-wh…
JSchmie Oct 11, 2024
81fefd5
Merge pull request #132 from JSchmie/dependabot/pip/openai-whisper-20…
JSchmie Oct 11, 2024
3fe1380
Merge pull request #134 from JSchmie/fix-audio-torch-device-setting
mahenning Oct 24, 2024
9528468
Merge pull request #128 from JSchmie/roll-back-torch-verision
mahenning Oct 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 95 additions & 0 deletions .github/workflows/docker.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# This workflow uses actions that are not certified by GitHub.
# They are provided by a third-party and are governed by
# separate terms of service, privacy policy, and support
# documentation.

# GitHub recommends pinning actions to a commit SHA.
# To get a newer version, you will need to update the SHA.
# You can also reference a tag or branch, but the action may change without warning.

name: Publish Docker image

on:

push:
tags:
- v*

workflow_dispatch:

env:
image: hadr0n/scraibe

jobs:
push_to_registry:
name: Push Docker image to Docker Hub
runs-on: ubuntu-latest
permissions:
packages: write
contents: read
security-events: write
steps:
- name: Check out the repo
uses: actions/checkout@v4
with:
fetch-tags: true
fetch-depth: 0

- name: Get Version Tag
id: version
run: |
echo "tag=$(git describe --tags --abbrev=0)" >> $GITHUB_OUTPUT

- name: Overwrite label tag
run: sed -i 's/LABEL version=".*"/LABEL version="'${{ steps.version.outputs.tag }}'"/' Dockerfile

- name: Test name and tag
run: |
echo "${{ env.image }}:latest,${{ env.image }}:${{ steps.version.outputs.tag }}"

- name: Log in to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Build and push Docker image
id: push
uses: docker/build-push-action@v5
with:
context: .
file: ./Dockerfile
push: true
tags: "${{ env.image }}:latest,${{ env.image }}:${{ steps.version.outputs.tag }}"

- name: SBOM Generation
uses: anchore/sbom-action@v0
with:
image: ${{ env.image }}:latest

- name: Scan image
id: scan
uses: anchore/scan-action@v3
with:
image: ${{ env.image }}:latest
fail-build: false

- name: upload Anchore scan SARIF report
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: ${{ steps.scan.outputs.sarif }}

# - name: Inspect action SARIF report
# run: cat ${{ steps.scan.outputs.sarif }}

- uses: actions/upload-artifact@v4
with:
name: SARIF report
path: ${{ steps.scan.outputs.sarif }}

# - name: Generate artifact attestation
# uses: actions/attest-build-provenance@v1
# with:
# subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME}}
# subject-digest: ${{ steps.push.outputs.digest }}
# push-to-registry: false
44 changes: 11 additions & 33 deletions .github/workflows/pypi.yml
Original file line number Diff line number Diff line change
@@ -1,18 +1,14 @@
name: Publish Python 🐍 distribution 📦 to PyPI and TestPyPI

on:
pull_request_target:
branches:
- develop
types:
- closed
paths:
- scraibe/**
- pyproject.toml

push:
tags:
- 'v*.*.*'
branches:
- "develop"
paths:
- "scraibe/**"
- "pyproject.toml"

workflow_dispatch:
inputs:
Expand All @@ -27,13 +23,7 @@ on:

jobs:
Build-and-publish-to-Test-PyPI:
if: |
(github.event_name == 'workflow_dispatch' &&
github.event.inputs.test == 'true') ||
(github.event_name == 'pull_request_target' &&
github.event.pull_request.merged &&
contains(github.event.pull_request.labels.*.name, 'release')) ||
(github.event_name == 'push' && startsWith(github.ref, 'refs/tags/'))
if: github.event_name != 'workflow_dispatch' || github.event.inputs.test == 'true'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
Expand Down Expand Up @@ -72,28 +62,16 @@ jobs:
needs: Test-PyPi-install
runs-on: ubuntu-latest
if: |
always() &&
(( needs.Build-and-publish-to-Test-PyPI.result != 'failure' &&
needs.Test-PyPi-install.result != 'failure' ) &&
((github.event_name == 'workflow_dispatch' &&
github.event.inputs.publish_to_pypi == 'true') ||
(github.event_name == 'pull_request_target' &&
github.event.pull_request.merged &&
contains(github.event.pull_request.labels.*.name, 'release')) ||
(github.event_name == 'push' && startsWith(github.ref, 'refs/tags/'))))
always() &&
(( needs.Build-and-publish-to-Test-PyPI.result != 'failure' &&
needs.Test-PyPi-install.result != 'failure' ) ||
((github.event_name == 'workflow_dispatch' &&
github.event.inputs.publish_to_pypi == 'true')))
steps:
- name: Checkout Repository Tags
uses: actions/checkout@v4
if: github.ref == 'refs/heads/main'
with:
fetch-depth: '0'
branch: 'main'
- name: Checkout Repository (Develop)
uses: actions/checkout@v4
if: github.ref == 'refs/heads/develop'
with:
fetch-depth: '0'
branch: 'develop'
- name: Set up Poetry 📦
uses: JRubics/[email protected]
with:
Expand Down
38 changes: 18 additions & 20 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#pytorch Image
FROM pytorch/pytorch:1.11.0-cuda11.3-cudnn8-runtime
FROM pytorch/pytorch:2.3.1-cuda12.1-cudnn8-runtime

# Labels

Expand All @@ -14,33 +14,31 @@ LABEL url="https://github.com/JSchmie/ScrAIbe"

# Install dependencies
WORKDIR /app
ARG model_name=medium
#Enviorment Dependncies
ENV TRANSFORMERS_CACHE /app/models
ENV HF_HOME /app/models
ENV AUTOT_CACHE /app/models
ENV PYANNOTE_CACHE /app/models/pyannote
#Enviorment dependencies
ENV TRANSFORMERS_CACHE=/app/models
ENV HF_HOME=/app/models
ENV AUTOT_CACHE=/app/models
ENV PYANNOTE_CACHE=/app/models/pyannote
#Copy all necessary files
COPY requirements.txt /app/requirements.txt
COPY README.md /app/README.md
COPY models /app/models
COPY scraibe /app/scraibe
COPY setup.py /app/setup.py

#Installing all necessary Dependencies and Running the Application with a personalised Hugging-Face-Token
RUN apt update && apt-get install -y libsm6 libxrender1 libfontconfig1
RUN conda update --all
#Installing all necessary dependencies and running the application with a personalised Hugging-Face-Token
RUN apt update -y && apt upgrade -y && \
apt install -y libsm6 libxrender1 libfontconfig1 && \
apt clean && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

RUN conda install pip
RUN conda install -y ffmpeg
RUN conda install -c conda-forge libsndfile
RUN pip install torchaudio==0.11.0+cu113 -f https://download.pytorch.org/whl/torch_stable.html
RUN pip install -r requirements.txt
RUN pip install markupsafe==2.0.1 --force-reinstall
RUN conda update --all && \
# conda install -y pip ffmpeg && \
conda install -c conda-forge libsndfile && \
conda clean --all -y
# RUN pip install torchaudio==0.11.0+cu113 -f https://download.pytorch.org/whl/torch_stable.html
RUN pip install --no-cache-dir -r requirements.txt

RUN python3 -m 'scraibe.cli' --whisper-model-name $model_name
# Expose port
EXPOSE 7860
# Run the application

ENTRYPOINT ["python3", "-m", "scraibe.cli" ,"--whisper-model-name", "$model_name"]
ENTRYPOINT ["python3", "-m", "scraibe.cli"]
Loading
Loading