With the release of the ElevenLabs API back in October 2023, the eleven labs service can be elevated to new hights.
The ElevenLabs API, just like any other service that uses APIs, is based on such API keys that are associated with the user’s account and, in this case, their ElevenLabs subscription. This means that when you generate any speech in your app with the ElevenLabs API key, your character quota is being used. While for some of us this idea isn’t worth it, for others it is. That’s mainly because you can use all of the generated voices that you generated using the voice design feature, all of the voices you imported from the voice library (A hub where people can share their generated voices only for all the world to see), but also, the best one for me honestly, the API also lends you access for using, in the apps it’s connected to, all of the cloned voices you got on your account.
Table of Contents
API access:
The API can be accessed by anybody who creates an ElevenLabs account, even with the free tier version. The steps are simple. Go to: https://elevenlabs.io/api, login with your account, then follow the steps to generate an API key, a string of characters that can then be pasted in any app that supports the ElevenLabs API.
App limitations and such:
The API is able to be used across as many apps as you want to, however only 2apps can generate speech at once, just like only 2users connected on your account can generate speech at once. For example, if you’re generating text in TechFreedom and @Voice at the same time, you can’t generate text in Legere Reader, until either Tech Freedom or @Voice is finishing it’s generation. This prevents the ElevenLabs servers from loading up, as there are, of course, a lot of users using the service alongside with you.
Current adoptions of ElevenLabs’ generation services: Why are they incomplete?
One of the reader apps mentioned earlier, Tech Freedom, has already got some support for ElevenLabs voices, but only the default ones. It most likely is using a custom API, as per the company’s request towards ElevenLabs. While this is indeed a welcome thing for some, there are a large number of ElevenLabs features this custom API key they’re using doesn’t support.
Do note that this statement is not to blame TechFreedom or ElevenLabs for their business choices, but simply to educate you about what you’re missing out on, and what a magic using our own API keys can unlock.
Before I’ll list the features you’re missing out on, remember that this is the first ever reader app for the blind to have support for any ElevenLabs generation of any kind, so take a moment to appreciate Tech Freedom’s current options for what they actually are. This first move is a better than amazing start, but it paves the path for even more advanced developments from ElevenLabs to come.
So here is, from some of the most important to the preferential ones, the list of features Tech Freedom’s ElevenLabs voices implementation lacks on:
- No voice parameters configuration.
On the ElevenLabs website, but also through it’s official API, you can get all the features they release for use in your apps. One of them is customizing voice settings/parameters. Not all the content you give is the same. Some content, for a maximum and flagship reading experience, needs to be given to a voice that knows best to voice that text. With AI voices, this is even more important, now when the realism in TTS voices has grown this much. The settings available are: Stability – Setting this parameter to a higher value will make the expressivity more variable, but is more prone to tonal mistakes. Simularity – This controls how different is the tone between sentences of the same kind: For example, if this is set to a higher number than 0, the voice may speak2 sentences of a calm person with slightly different tones. The highlighting of important words is also more probable. If it’s set to 0, however, there will be no such a big difference between a sentence and another. It’s important not to exaggerate, too much of anything is harmful, more so in this case. The last parameter you can deal with is style exaggeration. This is supposed to make the AI less likely to speak the text in a wrong tone, but turning this to anything destroys expressivity, it doesn’t work too much.
Tech freedom’s voices are unable to change these parameters. Should something go wrong and you need tweaking, you’re stuck and can’t do anything about it.
- No support for all of the premade ElevenLabs voices. TechFreedom’s voice list is very small compared to the list of default voices available in the official versions of ElevenLabs. Absolutely all of these voices are available even for the free tier, which is an amazing fit to look for from ElevenLabs’ side.
- No generation models. ElevenLabs supports more generation models. For now, here they are: Eleven Multilingualv1 and v2. V1 supports just 6languages, V2 supports more than 29. Eleven English V1 and V2. No language switching support, but V2 has a completely new tone detection and interpretation. Eleven Turbo. This model, while only available for English for now, is the fastest model available in ElevenLabs. While the multilingual and English models (V2) generate 5000characters in about 1minute and a half, Turbo generates them in less than 15seconds. Definitely an amazing model.
As per TechFreedom in this case, I haven’t checked or come to a conclusion for which model it supports, but either way, the models may or may not be updated as ElevenLabs improves them.
- No custom voices, at all. As we’ve mentioned before, ElevenLabs, through it’s API key, gives you access, across apps, to absolutely any voice you have on your account. Generated voices using Voice Design, imported ones through the voice library, cloned ones, even business voices, for who has paid for and has recorded for one.
Again, these features might or might not be useful to all of us. However the choice can be implemented. To be honest, for me at least, these features are really important. Some of you may have specific prefferences to voices. One clear example is me. I preffer and would really like to read with a raspy female voice. While this is just a random example, it shows that the ElevenLabs API, if coupled with a subscription, can really unlock powerful tools.
What can we do?
The more people suggest this to the developers of apps, the more likely it’s for them to adopt the idea. Using and configuring the API is a onetime process, at least until the device is reset. It’s not hard to use either. So after you have added your API key, there’s nothing left for you to do but maintain your subscription to ElevenLabs, if you want a decent amount of characters to use for reading and the ability to generate, import and clone voices.
After talking to the @Voice aloud reader developers about this, I’ve been told that if enough people ask for this feature, then it will be implemented.
All we can do is send imails to the support teams of each of these reader apps and pray, hopefully they’ll implement the addition of the ElevenLabs API keys, elevating our reading experience to unseen levels.
Comments