Elevenlabs.io is the current leader in the world of AI voice technology. We did an overview of its offering here, covering the various use cases for ElevenLabs’ tech. In this article, we’re going to focus particularly on the Voice Cloning tool. The Voice Cloning tool allows users to upload samples of a voice, and ElevenLabs will attempt to clone that voice, which can be used to generate new speech.
For my example, I created a fictitious debate between Joe Biden and Donald Trump about “The Vampire Problem”. In this article, I’ll explain how I made it, and how you can use the Voice Cloning tool to generate similar audio. Note that you’ll need a paid account to use the Voice Cloning tech.
Absurd AI Debate: Joe Biden vs Donald Trump on the topic of “The Vampire Problem”
Obviously this is a work of parody. To generate the audio, I used public domain samples of Trump and Biden speeches as source material, and used ChatGPT to create the script of this silly debate:
Now let’s go through the process of generating this debate audio.
How to Use ElevenLabs’ Voice Cloning Tool
Step 1: Get Your Source Audio
In order to clone a voice, you need source audio, ideally more than 5 minutes. If you are cloning your own voice, you can simply speak into a microphone or your phone. If you’re cloning other voices, you can source audio from YouTube, films, Podcasts, etc.
Just beware that there may be some issues with copyright, which we’ll go over later.
In my example, I sourced audio from Biden and Trump speeches from the Whitehouse.gov site and the Trump Admin Whitehouse Channel. These audio clips are in the public domain, and the output is clearly parody.
I used the free program Audacity to record audio from the speech videos.
Tips on Source Audio:
- Try to get around 5 minutes of audio
- Try to get audio of just the person speaking, without background noise, music playing, etc.
- Higher quality audio is more important than more data, but it depends what you want to create.
Step 2: Upload The Audio Files to ElevenLabs Voice Lab
Sign into your ElevenLabs account and go to the Voice Lab page. Click “Add a Generative or Cloned Voice” (note: you cannot do this on a free account). The following menu will pop up:
Choose “Instant Voice Cloning”. A new menu will pop up. Attach the audio sample you’ve created, and provide a name for the new voice. You can also enter labels, a description, etc.
You will also have to check off a box stating that you have permission to use this voice, and that you’re not going to be using it for nefarious purposes. More on this later.
Step 3: Check Your Newly Generated Voice
You should now see your new voice on your VoiceLab voice list. If you click on it, you will be able to enter text, and ElevenLabs will generate audio based on your text, in the voice you’ve created.
Fun times ahead!
Step 4: Generate The Text You Want to Use
In my case, for my vampire debate, I used ChatGPT to create a script for this outlandish debate. You can use whatever you want, or simply type up the words you want the cloned voice to say.
Step 5: Generate the New Audio using the VoiceLab voice
If you’re creating audio just for one voice, this is pretty simple. Enter in the text you want spoke, let ElevenLabs do its thing, and click the “download” button to download an mp3 of your new voice.
In my case, I had three voices: Trump, Biden, and a moderator (one of the premade ElevenLabs voices). Consequently, I had to skip back and forth between speakers to simulate the debate.
To stitch these together, I again used Audacity. It allows you to move audio samples around on a timeline, which I did to string the series of debates together into one audio clip.
Permissions and Copyright Concerns
As mentioned above, ElevenLabs requires that you certify that you have permission to use the voice sample, and that you will not use the generated voice for ill purposes.
I am not a lawyer, but my understanding of copyright is that the voices I used are in the public domain, and it is clearly parody, so it should fall as fair use.
But this tech is new, and the courts may view these generated voices in unexpected ways.
Do not use these voices for scams, to generate fake commercials to sell your products, or anything like that.
Final Thoughts
ElevenLabs voice cloning tool is really fun to play with, and is clearly very powerful. They have an even more powerful Professional voice cloning tool, but we have not yet used that — stay tuned!
We hope you come up with some creative uses for this tech. Just remember, you will need a subscription to ElevenLabs if you want to use these features. View ElevenLabs’ pricing here.
