Synchronized jaw movement with audio files (SOLVED)

I have a workaround!!!

T-800

I don't use AudioSignalProcessing, I read that it does not work very well and because I don't want the jaw to move with sound effects like beeps, music and tracking sound effects.

I had been investigating how to play the Terminator voice, and have jaw movement without success.
I thought I could play an audio and at the same time make it speak with volume=0 to have the jaw movement with fake text, but I don't know how to do it.

I'm not an experienced programmer, so I thought and thought and kept thinking. I started thinking this time with coffee because there was no more beer, until I saw how text to speech worked.

A cache wav file is created for the first time, therefore it can be replaced by another audio.

wav file generated

The extra word "baby" (Fake text) it's there to have a one more movement that fits with the audio.

The wav file is generated in the audioFiles/MarySpeech folder of the voice that is being used, in my case Spike:

I make a copy of the Terminator voice file:

wav file

 

Then I rename the file with the name that had been created by myrobotlab:

rename file

Now copy and replace the file created with our new file:

Now you can test it. You should hear the voice of terminator with jaw movement.

Test

 

Now it can simply be used in the aiml files:

<category><pattern>BY BYE</pattern>
    <template>
        hasta la vista baby baby
    </template>
</category>

or in the scripts:

i01.mouth.speak("hasta la vista baby baby")

And you will hear the Terminator voice with the movement of the jaw like any normal text.

If you have silences in the audio, the text should simulate them too:
desire... is... irrelevant... i am... a machine, a machine
Again, I had to put "a machine" twice to have an extra movement that fits with the audio.

Here is a video, I still need to adjust the limits of the jaw to make the movement better and maybe do it again by adding another "a machine". Remember that changing the text generates a new file.
 

I hope this could be useful to those of us who were looking to do this.

Hasta la vista Baby!

 

GroG's picture

Wow .. Looks great astro

Wow .. Looks great astro !

Nice description of your work-a-round too.  Always nice to see your updates.