AI Tool: Sora
Like many, I have been eagerly waiting for access to the latest generative AI tool from OpenAI called Sora - a tool that creates video based on text prompts. The hype over the release of this tool has been pretty big and the posted demos on the internet have been stunning. OpenAI announced last week that Sora was finally being launched to the public - but only for paid subscribers. At first, most folks received a message that Sora wasn’t adding new accounts as they were overwhelmed with requests, but I finally got in this past weekend. What I will say up front is that I’m not that impressed with the results so far, but I am quite certain that the end products will improve greatly over the next few weeks and months. I am sharing 5 videos in this post, along with the text prompts that generated them, to show you what Sora is doing now. I also share some thoughts on how this incredible tool might be incorporated into music education. Here is the sizzle reel that has been on YouTube for the past few months:
When you initially enter your text prompt, Sora creates a 5-second video clip. You can then go in an edit that clip in storyboard mode, similar to something like iMovie. There are multiple controls that you can use, including duration, variation, aspect ratio, resolution, and stylistic presets. The storyboard section looks like this:
For my very first text prompt, I entered the following: Create a video of a big band playing in a jazz club in Harlem in the 1920s. Include dancers and customers. Sora created two 5-second clips. Below is one of them, as well as the end product of the storyboard that I used.
If you look closely, you’ll see LOTS of weird things, including weird musical instruments (what kind of trump-bass is that??) and strange dance positions - not to mention the uncomfortable close up. Is it an incredible display of generative AI capabilities? Yes. Is the video anything that would convince someone that it wasn’t created by an AI bot? No. It’s pretty trippy in my opinion.
Here’s another one. In this video, I entered the following text prompt: Can you create a video of Mozart performing with an orchestra in 18th century Vienna? This is what Sora created:
Better for sure - but still weird. Here’s another one. Text prompt: Can you create a video of musicians in India performing on traditional music instruments?
Even better, but what is the guy second from the right doing? Air tabla-ing??? Weird.
Here’s another. Text prompt: Can you create a video of a music teacher teaching a group of elementary music students about the fundamentals of music?
What is that girl in the red t-shirt doing with her hands? Bless…
Here’s one last one. Text prompt: Create a video of a high school choir director conducting a difficult piece of music at a Spring concert.
That poor teacher looks like she needs a hip replacement very soon and that high school choir sure is homogenous!!
So, how can you use this tool with your music students? Personally, I think you should probably wait a while before you do, as well as consider whether the $20/month price tag is worth it. I am quite sure that Sora will improve very soon, and when it does, here are 5 simple suggestions for how you might use it in your music classroom:
Visualizing Musical Concepts - use Sora to create videos that visually represent abstract musical concepts, such as harmony, rhythm, or dynamics. For instance, generate animations that depict how sound waves interact to form harmonies or how rhythmic patterns are constructed. Again, you’ll probably have to wait on this to get useful results.
Historical Reenactments of Musical Eras - use Sora to produce short videos that recreate significant moments in music history, such as a Baroque-era concert or a jazz performance from the 1920s.
Student-Created Music Videos - have students use Sora to create their own music videos by inputting lyrics or themes and generating corresponding visuals.
Instrument Demonstrations - use Sora to generate videos demonstrating various musical instruments, including their sounds and playing techniques.
Animated Music Theory/Fundamental Tutorials - create animated tutorials that explain music theory concepts, such as scales, chord progressions, or song structures.
I know that I will be playing with Sora over the holidays. I look forward to this tool improving, but until it does, I am going to have a lot of laughs!