I asked AI to make a Music Video... the results are trippy
4,169,547
Published 2022-05-17
After getting caught up in the excitement around DALL·E2 (latest and greatest AI system, it's INSANE), I searched for any way I could use similar image generation for music synchronization. Since DALL·E2 is not available to the public yet, my search led me to VQGAN + CLIP (Vector Quantized Generative Adversarial Network and Contrastive Language–Image Pre-training), before settling more specifically on Disco Diffusion V5.2 Turbo. If you don't know what any of these words or acronyms mean, don't worry, I was just as confused when I first started learning about this technology. I believe we're reaching a turning point where entire industries are about to shift in reaction to this new process (which is essentially magic!).
Important note:
While this AI is impressive, it still required additional input beyond just the song lyrics to achieve the music video I was looking for. For example, I added keyframes for camera motion throughout the generated world. These keyframes were manually synchronized to the beat by me. I also specified changes to the art style at different moments of the song. Since many of the lyrics are quite non-specific, even a human illustrator would have a hard time making visual representations. To make the lyrics more digestible by the AI, I sometimes modified the phrase to be more coherent, such as specifying a setting or atmosphere.
This was my first time working with DDV5, and I'm very happy with the results! There were many times where my jaw dropped upon seeing what the AI came up with. I haven't felt this sense of wonder from technology since I first experienced a HD videogame as a child.
If you would like to learn more about how this video was made, try this yourself, or ask me any questions, I'll post a more detailed explanation of how to get started on Patreon (link below). The post is free to the public, no need to pay. If you do want to support me and become a member that would be much appreciated, you'll also automatically be entered into the end screen minigames where you earn points on each video and move up the leaderboard!
Join on Patreon to automatically have your name included in the next video: www.patreon.com/doodlechaos
Want to add lyrics and color beat blocks to your Disco Diffusion project like I did in my video? Here is my code: www.patreon.com/posts/67249569
My social media:
Twitter: twitter.com/doodlechaos
Discord: discord.com/invite/7FCrWAzDY7
Tiktok: www.tiktok.com/@doodlechaos
Shorts Channel: / @doodlechaosshorts2905
Instagram Shorts: www.instagram.com/doodlechaos_shorts/
Email: [email protected]
While Disco Diffusion is based on the contributions of many, show some love to the two most prominent contributors:
twitter.com/somnai_dreams
twitter.com/gandamu_ml
Music:
[Indie Dance] - Rezonate - Canvas [Monstercat EP Release] : • [Indie Dance] - Rezonate - Canvas [Mo...
All Comments (21)
-
If you would like to download my exact input settings, as well as links to all the resources I used to create this video for free, everything is on my last Patreon post: www.patreon.com/posts/i-used-ai-to-66518281 (Free for everyone, no need to become a member!)
-
POV: You're an artist trying to draw something, but you forget your idea every few minutes, so you have to try and make something out of your already existing shapes and colors
-
The seamless transitions from realistic images, sketches, and digital art are mind-boggling
-
It's amazing to think that if this came out just a few years ago before AI art it would win like ever single award.
-
I just thought that maybe, the reason our reaction to this kind of art is so strange, is because unlike humans, maybe this AI doesn't have a point of focus like us, So unlike human art or music videos, they're designed with the same understanding of drawing our attention to particular parts. But with this art, it's similar to meditation or psychedelics because you're in an open or free state where there's no focal point because you are still and free with no intention or in psychedelics case, ability to look at something with out it changing to quick to focus on, where you're kind of forced to go with. Because the AI's in the same state.
-
the style of this is like something between fantasy, cyberpunk, and paintings. its really interesting to see new landscapes appear out of the infinitely zooming camera view.
-
I love how my brain is constantly trying to identify objects and understand the 3d space but can't hold on to a single perception for more that a couple of seconds.
-
This is really well done. I really liked how extreme closeups marked the transitions between the different scenes, rather than using fades, dissolves, or cuts. It gives a feeling that the viewer is traveling along a road, instead of passively sitting and watching.
-
IT feels like those AI generated images where you think you know whats going on then you look closer than you realise its not what it seems
-
I feel like this video really reflects how our own dreams are built, taking pieces of words/thoughts and generating them into images that are an abstraction of them, still somewhat recognizable, but not fully graspable.
-
This looks like the things you'd see in a dream. Where everything doesn't makes sense but they just fit together nicely.
-
1:36 is definitely my favorite part, just the way the camera angle is turning and zooming in and changing, and how the music pairs with the visuals... incredible
-
Awesome!!! The details are moving and changing with the flow of the beat, every instrument gets visual attention and even the words in the song are finding an interpretation!
-
The best representation of "Don't let others know your next move.". The AI absolutely nailed it, dang im amazed.
-
This is absolutely incredible, but the thing that stood out to me was the AI taking the strings of text that you presumably overlaid, and turning them into words relating to art. I definitely didn't expect that.
-
Thank you very much for sharing and for being gracious enough to share the creative process too.
-
Love the caption, reflects my own excitement about AI when I first learned about it. I was so hooked!
-
I really like how at 4:00 the lyrics on the video say 'it's quite complex' but then it completely morphs to 'art's quite complex', such a cool touch
-
As an AI engineer, I want to point out something extremely cool to me that otherwise people might not know. All these images and moving through this surreal space.. it's basically exploring a multi-dimensional world, with all the AI remembers.. We're travelling through the AI's actual mind, experiencing the various imagery and 'impulses' it experiences as it interprets the words of the lyrics (or the artist's text. Probably the latter, it looks like DDv5 Turbo). It's like your brain when you're asleep, sorting memories. The fact this looks like we're moving through a perfectly linked continuous 3D space is just a feature of the AI where similar-looking things are connected, and we're morphing various 'alignments' of them.
-
This was the dopest thing I've seen in a while. Amazing work 🙌🏻✨
