Visualizing Music using AI tools

This project contains code and notes for AI music video generation. Using my old songs as source inspiriation.

Video Projects

• Endless Ever
  • https://www.youtube.com/watch?v=E_SUavxxSs8
• Sergeant
  • https://www.youtube.com/watch?v=QjFBfwnoZuM
• Buttercup
  • https://www.youtube.com/watch?v=a4JHBTHhb0w
• Drumwah
  • https://www.youtube.com/watch?v=z8TY0dtIzLw

Source / Attribution

List of projects used in these experiments.

StyleGan

• https://github.com/NVlabs/stylegan2
• https://github.com/NVlabs/stylegan3
• https://github.com/PDillis/stylegan3-fun

Models + Datasets

• https://github.com/justinpinkney/awesome-pretrained-stylegan2
• https://github.com/edstoica/lucid_stylegan3_datasets_models

Lucid Sonic Dreams

• Original: https://github.com/mikael-alafriz-deel/lucid-sonic-dreams
  • Well used fork: https://github.com/NotNANtoN/lucid-sonic-dreams
  • Parameter Notebook: https://colab.research.google.com/drive/1Y5i50xSFIuN3V4Md8TB30_GOAtts7RQD?usp=sharing
• Nerdy Rodent fork
  • https://github.com/nerdyrodent/lucid-sonic-dreams
  • https://www.youtube.com/watch?v=tdhiTL2NWSo
• Useful description of how LSD works
  • https://towardsdatascience.com/introducing-lucid-sonic-dreams-sync-gan-art-to-music-with-a-few-lines-of-python-code-b04f88722de1

Spleeter

• https://github.com/deezer/spleeter

ruDALL-E

• https://github.com/ai-forever/ru-dalle
• https://hive.blog/hive-158694/@dbddv01/making-your-own-model-for-ru-dalle-the-looking-glass-ai

Visual Loop Machine

• TODO: https://github.com/goolygu/v_machine

Stable Diffusion / ControlNet

• TODO: https://github.com/AUTOMATIC1111/stable-diffusion-webui
• TODO: https://github.com/Mikubill/sd-webui-controlnet
• Interesting gateway idea: https://www.youtube.com/watch?v=W5hDKeQU8Y8
  • https://civitai.com/models/21813/edge-of-realism

NLE: DaVinci Resolve

• https://www.blackmagicdesign.com/products/davinciresolve/

DAW: Reaper

• https://www.reaper.fm/

Samples / Plugins

• Spitfire Audio: BBC Symphony Orchestra Discover