Syncing a servo to audio

[mbeals 74]

I have an interesting problem I'm hoping someone can help me sort out. I'm building a talking pumpkin using a G2553 that will playback audio clips from a SD card with mouth movement synced to the audio.

 

To handle speech, I plan on hooking up an SD card and reading it with the thin_fat package (I can't post links but look up a post on this site called 'SD Media Player with FAT16'). I will output a PWM signal through a simple filter to a speaker. I'm hoping with this micro that I can hit 16 bit audio, but I could live with 8 bit if needed.

 

The part I'm having problems figuring out how to do is syncing the mouth to the audio. I thought about making a secondary file to accompany each audio file with the timings, but I'm not sure I can read two files at once like that, and I really don't want to sit there and make these files by hand. I want to be able to just resample some files and write them to the SD card and run with it.

 

So I've been thinking that the audio waveform would have dead time in the regions where the mouth is closed, but non-deadtime elsewhere. So if I could lowpass the crap out of the audio to eliminate the short pauses and variations and then convert it to TTL, I would be left with a binary mask dictating when the mouth should be open vs closed. I could then switch the waveform to the servo based on which 'mode' it should be in.

 

In theory it sounds good, but I'm not sure the best way to implement this. To do it all on the micro, I believe I would need to buffer quite a bit on the chip, and I don't know if I have enough memory to pull off. I've considered doing this off chip with an RC low pass filter and schmidt gate, but that doesn't solve the delay issue. Maybe if I use a second micro dedicated to the audio processing I could salvage enough memory to buffer it?

[jsolarski-backup 22]

I like this Idea, but My take on it would be one msp to do the audio, and the other to move the mouth.....

I would base the movement on volume, the louder the sound the bigger the mouth would open, so breaks in speech would close the mouth. But I think it could be done with one chip.

[mbeals 74]

interesting....I never considered allowing the mouth to be in variable positions. I could feed the filtered signal back to an adc and use that to scale the pwm. thanks!

[russcky 9]

Hi @@mbeals, I'm trying to add audio off an SD card as well to my project.  Just checking into seeing if your project was successful and what you can share that might be helpful?

 

I'm using a Nokia 5110 LCD with images on it that I want to sync to audio at times.

I'm using Energia and have the LCD pulling images off the SD card, but trying to find something similar to this Arduino project (http://hlt.media.mit.edu/?p=1963) where they are able to playback audio clips at will.

 

Any suggestions or ideas on the audio work you did would be greatly appreciated!  

[mbeals 74]

I actually didn't get all that far with the audio.  The memory on the g2553 just wasn't large enough to do what I wanted it to do, and I had to switch focus to other projects before I came up with a solution.  I do plan on coming back to this project, but I will be using an ARM over the MSP.

[russcky 9]

Thanx @@mbeals, I'll see what I can get working and post back here. 

[rockets4kids 204]

The MSP430 isn't capable of PWMing much more than 8-bit audio, and you aren't going to hear the difference in a small speaker anyways.  Although you could filter the audio on the MCU, it would be much easier to pre-process all your audio on the host and store it on the SD card along with your audio.  The SOX audio library will be your friend here.

[russcky 9]

@@rockets4kids, I have Audacity already, will SOX audio library provide even more capabilities helpful to improving the quality of the sound produced?

 

Trying to get the best sound possible for mono speech and some music (AM radio quality is fine, just need minimal noise).