Following yesterday’s post about a brief history of text to speech, today we list some of the techniques involved in creating speech synthesis.

Articulatory synthesis

In an articulatory synthesis, models of the human articulators (tongue, lips, teeth, jaw) and vocal ligament are used to simulate how an airflow passes through, to calculate what the resulting sound will be like. It is a great challenge to find good mathematical models and therefore the development of articulatory synthesis is still in research. The technique is very computation-intensive but memory requirements is almost nothing.

Formant

The synthesis is a sort of source-filter-method that is based on mathematic models of the human speech organ.
The approach pipe is modelled from a number of resonances with resemblance to the formants (frequency bands with high energy in voices) in natural speech.
The first electronic voices Voder, and later on OVE and PAT, were speaking with totally synthetic and electronic produced sounds using formant synthesis. As with articulatory synthesis, the memory consumption is small but CPU usage is large.

Concatenating synthesis

A concatenating synthesis is made of recorded pieces of speech (sound-clips) that is then unitized and formed to speech. Depending on how long sound-clips that are used it become a diphone or a polyphonic synthesis. The later in a more developed version is also called a Unit Selection synthesis, where the synthesizer has access to both long and short segments of speech and the best segments for the actual context is chosen.

Diphone

For a diphone synthesis the elements from the recorded speech are very small.
The strength in this case is that almost any sentence or expression may be read but quite often there are errors in the pronunciation and if the model used for prosody is not good, or modelling is difficult, the speech may sound a bit monotonic.
A diphone synthesis doesn’t work that well in languages where there is a lot of inconsequence in the pronunciation rules (English, Swedish etc) and in special cases where letters is pronounced differently than in general. The diphone works better for languages that have large consistencies in the pronunciation (Spanish, Finnish etc.) Another advantage is that the prosody, the intonation, can be described in very much detail.

Unit selection

The greatest difference between a Unit selection and a diphone voice is the length of the used speech segments. There are entire words and phrases stored in the unit database. this implies that the database for the Unit selection voices are many times bigger than for diphone voices. Thus, the memory consumption is huge while the CPU consumption is low.

The most important issue is to still get a natural and smooth prosody. This is hard because the units contain both intonation and pronunciation since entire phrases are used almost directly from the recorded data. Since the first Unit selection voice was released, over eight years ago, there has been much improvements for each new voice with every release. This is by far the most widely used technique among our providers.

HMM synthesis

A quite new technology is speech synthesis based on HMM, a mathematical concept called Hidden Markov models. It is a statistical method where the text-to-speech system is based on a model that is not known beforehand but it is refined by continuous training. The technique consumes large CPU resources but very little memory. This approach seems to give a better prosody, without glitches, and still producing very natural sounding, human-like speech. We collaborate with providers offering this technique as well.

Customizations and improvements

On top of using the best voices available we also add our own layer of improvement, both general and customer specific customizations. We have linguists with long experience of speech synthesis working with transcriptions to tweak the pronunciation and reading of the spoken text. Therefore we can greatly help our customers that want to optimize the quality of the text to speech on their web pages. Sometimes it is enough to do a quality control of a couple of hours listening to your website and correct the errors we find. In other cases some of our customers have industry specific words (think of the pharmaceutical industry for example) where it is very important that they are pronounced correctly.

One of the largest customizations we have made so far was for a customer who sent us a list of over 3000 words that had to be quality controlled. Another customization was for a site with about 200 000 pages where the same acronym or abbreviation had to be expanded differently depending on at what part of the site it was mentioned in. Many users wonder why the same voice reads so much better when it is used in our services compared to when the same voice, or text-to-speech system, is used for reading similar, or the same, content with other softwares or services. The answer is the above mentioned customizations.

Thanks to Professor Hartmut Traunmüller, Dept. of Linguistics at the University of Stockholm for a lot of the facts, the picture and the sound samples on this page.

Behind all our services there is a server-based software performing the speech synthesis, called text-to-speech software. The voices we use are provided by different providers but the technique behind the different voices has many similarities. Therefore we like to tell you briefly about the development of speech synthesis and its history.

The history of speech synthesis

Over the last few years there has been a great development of the quality of the speech produced with text to speech. Many people think that synthetic speech as it is also called sounds like robots from older movies. The truth is though that some voices almost sound like recorded speech and due to that we have seen a very strong growth of user groups for our services the last years.
When we invented the talking web in 2001 the target group was people with reading difficulties but now we see that the user group is much broader.

What you maybe don’t know is that the first synthetic speech was produced as early as in the late 18th century. The machine was built in wood and leather and was very complicated to use generating audible speech. It was constructed by Wolfgang von Kempelen and had great importance in the early studies of Phonetics. The picture to the right is the original construction as it can be seen at the Deutsches Museum (von Meisterwerken der Naturwissenschaft und Technik) in Munich, Germany.

Here’s an audio sample of the synthetic speech the machine produced (WAV-file 776 kB) .

(First there is a human that says a sentence and then the machine tries to say the same. This was made by a re-construction of Kempelens machine.)

In the early 20th century when it was possible to use electricity to create synthetic speech, the first known electric speech synthesis was “Voder” and its creator Homer Dudley showed it to a broader audience in 1939 on the world fair in New York.

Here’s an audio sample of Voder, the first electronic speech synthesis ever (WAV-file 381 kB)

One of the pioneers of the development of speech synthesis in Sweden was Gunnar Fant. During the 1950s he was responsible for the development of the first Swedish speech synthesis OVE (Orator Verbis Electris.) By that time it was only Walter Lawrences Parametric Artificial Talker (PAT) that could compete with OVE in speech quality.

Here’s a sample of OVE speech synthesis (WAV-file 77 kB).

and here’s a sample of the PAT speech synthesis (WAV 117 kB).

OVE and PAT were text-to-speech systems using Formant synthesis.

Speech synthesis becomes more human-like

The greatest improvements when it comes to natural speech were during the last 10 years. The first voices we used for ReadSpeaker back in 2001 were produced using Diphone synthesis. The voices are sampled from real recorded speech and split into phonemes, a small unit of human speech. This was the first example of Concatenation synthesis. However, they still have an artificial/synthetic sound. We still use diphone voices for some smaller languages and they are widely used to speech-enable handheld computers and mobile phones due to their limited resource consumption, both memory and CPU.

It wasn’t until the introduction of a technique called Unit selection, that voices became very naturally sounding. this is still concatenation synthesis but the used units are larger than phonemes, sometimes a complete sentence. We use different providers for different languages to always assure we can offer the best voices available for that language.

In a next post we will cover the different techniques behind speech synthesis.

Thanks to Professor Hartmut Traunmüller, Dept. of Linguistics at the University of Stockholm for a lot of the facts, the picture and the sound samples on this page.

A sausage says more then a thousand words.

Scan, one of the leading Swedish brands just launched a really great viral marketing campaign using VoiceCorps SpeechMachine solution. The idea for the campaign is quite cool. The campaign is for marketing Scan’s new line of spicy sausages. They wanted to add some nice interaction with the user so they added the strongest media around. Speech.

The core functionalty is that the users can send “speech-cards” to each other. They enter the text, listens if it is good and send the speech card to a friend.

The cool thing is that we used a Spanish voice but using Swedish speech rules. The result is a Spanish guy speaking Swedish. It’s brilliant! It really sounds like a guy from Spain that only lived a few years in Sweden. Enough time to learn the language but keeping a strong Spanish accent. The speech solution itself was delivered in just a couple of hours thanks to SpeechMachines ability to integrate with all the TTS engines on the market.

SpeechMachine is provided by VoiceCorp as a 100% hosted service that allows creative web developers to easily add text-to-speech functionality to their web apps without requiring any knowledge about text-to-speech technology. The communication with the customer’s web based app and the SpeechMachine is based on standard HTTP requests, and is therefore really easy to integrate in any web app.

Want to try out the app, http://www.scan.se/kryddigakorvar/

I have for the last 8 years worked developing SaaS solutions that in different ways speech enable web content. The business logic is quite straight forward; the customers (content owners) are the ones subscribing to and paying for the product and their visitors are the ones using it.  So, it is a service that the website owners offer to their visitors.

There are other ways for the visitors to listen to the content as well, even without the need for the website owner to subscribe to any service. Either the user can download any of the large number of software available at places like download.com or simply use the built in text-to-speech engine available in almost all existing operating systems. This is nothing new, it has been built in on Mac’s for decades and in Windows from W95. I remember my Amiga 1000 had it as well when I was a child. However, the quality of the voices available for free is not just THAT fantastic, but for many people; good enough. It really depends on how badly you need it.

People that depend on text-to-speech in order to use their computer at all already have a solution installed on their PC. They need it from anything to start MS Office, or start the web browser. These we can call “Professional users”. They have certain demands and requirements on the text-to-speech that differ quite a lot from people that do not depend on it. These professionals I am talking about are severe visually impaired or blind. They normally use software and equipment that costs tens of thousands of Euros. Pretty unreachable if they didn’t get support from the government.

Then we have a really large group, almost 20% of the population in most European countries that have milder difficulties with text. These are for example people with reading difficulties, dyslexia, low literacy level or are not native speaking. These groups have proven to be greatly helped by having the text read to them. They have other requirements on the text-to-speech voice, and they usually would need to buy the software themselves if they want a higher quality than the free ones. A good TTS for personal use can cost anything between 100 to 1500 Euros. Most people can not afford this.

The even bigger group of people these days that have proven to appreciate text-to-speech is “all the others”. Listening to content online like the newspaper, their email, reports etc enables them to be more efficient. To be able to perform other tasks at the same time as they read like driving a car, doing the dishes, commuting for example. These people have other requirements on the TTS. For example; It should sound very human. And there are no TTS products that target these groups of people.

The explosion of Audio Book consumption all over the world shows that for many people, even those without any reading or visual problems prefer to listen. It is just a fact. Then, the people that are depending on speech version of printed text are the true winners! Design for all is just fantastic!

I think that together with publishing lots of text follows a responsibility; a responsibility to make it accessible to as many as possible. And I do not need to say that making text content talk helps a whole lot of people in various situations.

Back to the SaaS track. The Products I have been working with do just this. We are speech enabling the web. This enables the customers to give their visitors/users a service and support. Today millions of people using our services every month, and that proves that it is a well adopted feature. Web based services need not only to be accessible and user friendly. They in fact also need to be “use worthy”.

It is like when you go to the supermarket. They usually have a free service to offer you a piece of assistive technology known as “a shopping cart”. This is to help people overcome their handicap of not being able to carry a lot of grocery goods at the same time. It is great. Everybody uses them if they intend to buy more stuff than they can carry. It helps the customers to buy more. This is a service. It is use worthy.

You do not need to buy (or build) your own shopping cart to go mass shopping since this is a supportive service the supermarket offers.

There are a couple of  “non SaaS” suppliers of Speech software.

Some thinking like this; “we let the users buy and download and install software on their PC so that they can listen to web pages”.

Others think like this; “We let the users download and install software on their PC for free, and then we bill the website owners that would like this software to be able to read their websites”

Those two are not services at all. It is simply a way to sell software products. The only difference is; who is paying.
 
I reason like this; the user/visitor would like to use the text to speech service without having to download anything. And the user should not need to pay anything to use it. And the user should be able to use this service from whatever device he likes; from a web browser on an Internet café or other public location, from his mobile phone, from whatever device and from wherever. The user can “carry” around his need to listen wherever he goes, and his need is not in any way tied to a specific individual computer where he has installed the software.

There are also others that try to do something in between; offering a limited web based speech service with the option for the user to download and install software for additional features that only works for a limited number of hours, then the user needs to re-install it to have it work for another few hours. I wonder how smooth solution that is in the long run.

No, the way to go is a totally server based product, where the best technology for producing high quality text to speech can be used, all improvements and updates are fully centralized and it is a device independent solution that works for anybody anywhere. Someone needs to pay, yes. It is pretty much the same logic as with the shopping carts; the content provider pays.

I met one of our TTS suppliers the other day. Lars-Erik Larsson, the CEO of Acapela Group. He told me about their latest development being a service to create corporate voices for their text-to-speech engine. The coolest thing was that they could now offer it at a very reasonable cost thanks to a new technology and procedures they have developed. The Acapela Voice Factory. and it only takes between 3 weeks(!) for the simplest version and up to 14 weeks for the full quality version!

Still the cost level is not really reachable for private users, but there are other (free) alternatives as well like the FestVox with the Festival TTS platform (however you can not in any way compare the quality with the commercial solutions).

With a price tag starting at 7500€ (excluding the cost of the speaker) it is really reachable for a lot of companies that want to have a corporate TTS voice that can easily pay off by using TTS in to automate some customer support, automate switchboards, or just do it as a fun thing in for example interactive web campaigns. But you would also need to buy licenses for the engine itself if you want to use it. But that is a very reasonable cost in such a project.

Is there a market? Sure. Lets say we have two car manufacturer that want to integrate speech into their cars. Obviously, Volvo would not want to use the same voice as Saab for instance. You must hear the difference . Some companies today have a corporate voice that they use in all radio and TV commercials, and wouldn’t it be great to have that voice talent also answering the phone on 30 lines at the same time?

However, if you have dreamt of immortality, this is one step closer. But if you decide to give up your voice up to a TTS engine, there are a couple of things you should be aware of.

• Your voice can technically be used to speak very dirty words and there is always the risk of people using it in a very very bad way.
• You can never really use your voice as something that identify that it is actually you. I.e. quite a problem if you also happens to be a big fan of speech verification systems…

The target group for corporate voices are mainly, well, corporations. Congratulations Acapela-Group!

Dow Jones have really understood what speech enabling is about. It is not just something you do to help people with various disabilities. It is so much more than that. This is true “design for all” where the people with different difficulties are the biggest winner. And on top of that, they use it as a competative advantage. That’s the spirit Dow Jones!

See Dow Jones press release below. / Niclas

Dow Jones Factiva Listen Capability Transforms the Way Users Consume News

Time-Saving Tool Enables Users to Perform Other Tasks While Listening to Relevant News
NEW YORK, (March 5, 2008) – Dow Jones & Company introduced a new “text-to-speech” capability in Dow Jones Factiva that allows users to listen to the news that drives their business. With one click, users can now listen to a news article rather than read it, freeing them to do other things and multitask as the pace of business today requires.

Currently available in beta format, a “Listen to Article” link appears at the top of any full-text article with fewer than 4,000 words. The listen capability is available in English, French, German, Italian and Spanish languages and automatically defaults to reading in the interface language previously selected by the user.

“Dow Jones Factiva continues to set itself apart from the competition by being the first to offer text-to-speech technology in the current awareness, news and research market,” said Dennis Cahill, senior vice president and chief product officer of the Dow Jones Enterprise Media Group. “This new capability builds on our commitment to provide customers with relevant news when, where and how they need it and to reinforce our No. 1 position in the marketplace.”

The listen capability is a Web-based service provided by VoiceCorp (www.voice-corp.com) that converts text into speech on the fly. It is made available wherever full-text articles are found, including alerts, search results and newsletters. Once the link is clicked, the listen capability uses a Flash player to read the article.

The addition of text-to-speech further builds on Dow Jones’s goal of integrating various forms of multimedia content into Dow Jones Factiva. In August 2007, Dow Jones Factiva added highly relevant video and audio information including business news, CEO interviews, executive speeches, shareholders meetings, product reviews and other meaningful business content. Dow Jones Factiva searches across more than 14,000 authoritative sources, including the exclusive combination of The Wall Street Journal, Dow Jones and Reuters newswires.

Ok, I have finally decided to start a blog.

It will be about my passions.

That being; Entrepreneurship, software as a service (SaaS), text-to-speech (TTS), web accessibility and the world of web business.  I have been in the industry since 1999 and have founded a number of companies and technologies on my way towards the future. In 1999 I together with some friends founded Phoneticom AB, a small (only me) company that was going to explore what audio in general and speech in particular could add to the user experience on websites. After only one year we launched the first ever server based web service for Text-To-Speech. ReadSpeaker was born. After a few years of setting up companies to sell the ReadSpeaker services, sell consultancy in web accessibility, creating new innovations etc I think I have finally landed in my new company VoiceCorp. 

My philosophies about;

SaaS
It is better to let one person make a proper job installing the software on a server to let other people just use it.

Web accessibility
This is broad. It’s not about making the web working for people with disabilities, it is way to reach out to a maximum number of people, nondependent of device used, technology skills or abilities.

Entrepreneurship
Either you are or you’re not, or maybe you are but you just haven’t found out just yet.

TTS
Using text-to-speech technology is a way to free text from letters and words. It is just great for all occasions where reading is just not a good option. The quality of the speech is more about how bad you need it rather then how good it really speaks.