Eleven Labs, a startup that offered voice synthesis by mimicking any voice, was shut down just days after its launch announcement

The just-launched audio deepfake services of a startup called Eleven Labs were hijacked for spreading racist and homophobic messages spoken using the voices of recognizable sights.

What could be worse than seeing your face plastered across the faces of people participating in deeply disparaging activities? The answer would be, to hear terrible things said by said protagonists, uttered using your own voice.

The company specializing in text-to-voice services announced its beta-testing program on January 23, with the developers promising to accurately reproduce the style and cadence of any human being’s voice using nothing more than a few audio samples sent to artificial intelligence-based software for analysis.

Dubbed Voice Lab, the service quickly attracted the attention of opportunists who did not hesitate to use it to mimic the voices of well-known celebrities, good at reciting the most outrageous phrases. For example, actress Emma Watson can be heard reciting passages from the famous book Mein Kampf, published by dictator Adolf Hitler to justify Nazi ideology. Another “telling” example is when Justin Roiland, the voice of Rick Sanchez from the animated series Rick and Morty, can be heard telling a good story about how he’s going to beat his wife badly, in reference to the unfortunate accusations against him in real life.

Crazy weekend – thanks to everyone for trying out our Beta platform. While we see our tech being overwhelmingly applied to positive use, we also see an increasing number of voice cloning misuse cases. We want to reach out to Twitter community for thoughts and feedback!

– ElevenLabs (@elevenlabsio) January 30, 2023

Apparently, the administrator of ElevenLabs failed to implement effective mechanisms to prevent abuse, with the AI-based service quickly being hijacked for content generation on a wide variety of topics.

Later, the company said on Twitter that they had had a “crazy weekend” and noted that while their technology “has been overwhelmingly applied to positive use,” the developers also saw a “growing number of voice cloning misuse cases,” without naming a specific platform. What is certain is that based on their experiences, developer Eleven Labs can expect costly lawsuits that could abruptly halt their ambitions for global fame.