AI music video. "In this video, I utilized artificial intelligence to generate an animated music video for the song Canvas by Resonate". Made with a software program called Disco Diffusion V5.2 Turbo, which apparently uses a technique called VQGAN + CLIP, which stands for Vector Quantized Generative Adversarial Network and Contrastive Language-Image Pre-training.
"While this AI is impressive, it still required additional input beyond just the song lyrics to achieve the music video I was looking for. For example, I added keyframes for camera motion throughout the generated world. These keyframes were manually synchronized to the beat by me. I also specified changes to the art style at different moments of the song. Since many of the lyrics are quite non-specific, even a human illustrator would have a hard time making visual representations. To make the lyrics more digestible by the AI, I sometimes modified the phrase to be more coherent, such as specifying a setting or atmosphere."
There are no comments yet.