Skip to content

Commit

Permalink
1. Rewrote script in Python using stdlib modules, so no pip modules a…
Browse files Browse the repository at this point in the history
…re required.

2. Store and use CivitAI API token to download models.
3. Show progress bar when downloading the model.
  • Loading branch information
ashleykleynhans committed Mar 27, 2024
1 parent 6ee42d6 commit fea6d47
Show file tree
Hide file tree
Showing 4 changed files with 129 additions and 32 deletions.
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
venv
.idea
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
# Bash script to download models from CivitAI using curl
# Python script to download models from CivitAI using an API token

## Installation

```bash
git clone https://github.com/ashleykleynhans/civitai-downloader.git
mv civitai-downloader/download.sh /usr/local/bin/download-model
mv civitai-downloader/download.py /usr/local/bin/download-model
chmod +x /usr/local/bin/download-model
```
## Usage

```bash
download-model [URL] [DESTINATION]
download-model [URL]
```

eg:

```bash
download-model https://civitai.com/api/download/models/15236 /workspace/stable-diffusion-webui/models/Stable-diffusion
download-model https://civitai.com/api/download/models/15236
```

## NOTE
Expand All @@ -25,4 +25,4 @@ download-model https://civitai.com/api/download/models/15236 /workspace/stable-d
user. If not, the installation commands should be prefixed
with `sudo`.
2. It is important to ensure that you use the **DOWNLOAD** link
and not the link to the model page in CivitAI.
and not the link to the model page in CivitAI.
122 changes: 122 additions & 0 deletions download.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
#!/usr/bin/env python3
import sys
import argparse
import urllib.request
from pathlib import Path
from urllib.parse import urlparse, parse_qs, unquote


CHUNK_SIZE = 1638400
TOKEN_FILE = Path.home() / '.civitai' / 'config'


def get_args():
parser = argparse.ArgumentParser(
description='CivitAI Downloader',
)

parser.add_argument(
'url',
type=str,
help='CivitAI Download URL'
)

return parser.parse_args()


def get_token():
try:
with open(TOKEN_FILE, 'r') as file:
token = file.read()
return token
except Exception as e:
return None


def store_token(token):
# Ensure the directory exists
TOKEN_FILE.parent.mkdir(parents=True, exist_ok=True)

# Write the token to the file
with open(TOKEN_FILE, 'w') as file:
file.write(token)


def prompt_for_civitai_token():
token = input('Please enter your CivitAI API token: ')
store_token(token)
return token


def download_file(url, token):
# Prepare the initial request with necessary headers
headers = {
'Authorization': f'Bearer {token}',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3',
}
request = urllib.request.Request(url, headers=headers)

# Disable automatic redirect handling
class NoRedirection(urllib.request.HTTPErrorProcessor):
def http_response(self, request, response):
return response
https_response = http_response

opener = urllib.request.build_opener(NoRedirection)
response = opener.open(request)

if response.status in [301, 302, 303, 307, 308]:
redirect_url = response.getheader('Location')

# Extract filename from the redirect URL
parsed_url = urlparse(redirect_url)
query_params = parse_qs(parsed_url.query)
content_disposition = query_params.get('response-content-disposition', [None])[0]

if content_disposition:
filename = unquote(content_disposition.split('filename=')[1].strip('"'))
else:
raise Exception('Unable to determine filename')

response = urllib.request.urlopen(redirect_url)
else:
raise Exception('No redirect found, something went wrong')

# Download the file
total_size = response.getheader('Content-Length')
if total_size is not None:
total_size = int(total_size)

with open(filename, 'wb') as f:
downloaded = 0

while True:
buffer = response.read(CHUNK_SIZE)

if not buffer:
break

downloaded += len(buffer)
f.write(buffer)

if total_size is not None:
progress = downloaded / total_size
sys.stdout.write(f'\rDownloading: {filename} [{progress*100:.2f}%]')
sys.stdout.flush()

sys.stdout.write('\n')
print(f'Download completed. File saved as: {filename}')


def main():
args = get_args()
token = get_token()

if not token:
token = prompt_for_civitai_token()

download_file(args.url, token)


if __name__ == '__main__':
main()
27 changes: 0 additions & 27 deletions download.sh

This file was deleted.

0 comments on commit fea6d47

Please sign in to comment.