Jump to content
Gulam

Project f-Spell: Now you can type even without Keyboard!

Recommended Posts

It all started with an aim of improving communication between speech & hearing impaired and normal people. But it has now almost become an alternative input device for any computing device!

 

So what is it all about? It's a cyber glove that recognizes sign language gestures and convert them into ASCII. Voila! We can throw away our keyboards and start using this glove. Right? After all, our keyboard does the same thing.

 

Don't try too hard to visualize this peculiar device! This is how it will be!!

 

post-3-0-07202000-1399253106_thumb.png

And Technically it will be something like this

 

And more technically the flow of data will be like this

post-3-0-09800000-1399253105_thumb.png

 

 

Got a overall picture? Okay. Let's see how this is implemented. The sensor boards which are mounted on the finger tips is a custom PCB (see attachements schema_acc_board.png & pcb_footprint_acc_board.pdf) which houses the accelerometer (in our case MMA7361L). Those accelerometers will be connected MSP430 Launchpad and Launchpad will then be connected to Bluetooth Module as shown below. (BT boosterpack from iTeadStudio is used here)

 

post-3-0-93485400-1399253106_thumb.png

 

Wondering why not all axes of accelerometers are utilized? Well we have to do some trade-off between resolution and no. of ADC channels available :-( 

 

That's all. Find the code as attachment (code.zip)

 

Here are few pics of the setup & output...

 

 

 

post-3-0-28242700-1399253102_thumb.jpg

post-3-0-70524400-1399253103_thumb.jpg

post-3-0-49630700-1399253104_thumb.png

 

Attempts have been made to miniaturize the whole setup into one glove. The FET Debugger is not able to recognize the custom board. But I'm not sure whether the problem is with Schema or PCB footprint.  I've attached the unverified schema and footprint also with this post. Interested are welcome to give a try! If you come up with a better design you are welcome to share it with me at any time ;-)

 

 

pcb_footprint_acc_board.pdf

post-34425-0-75014700-1398610591_thumb.png

code.zip

pcb_1_NOT_VERIFIED.pdf

pcb_2_NOT_VERIFIED.pdf

pcb_3_NOT_VERIFIED.pdf

schema_NOT_VERIFIED.pdf

Share this post


Link to post
Share on other sites

@Gulam This is cool. Thank you for sharing!

Are you looking at accelerometer patterns. How do you know what letter to select?

@@bluehash Yes! Accelerometer values of each finger will be normalized/equalized (to avoid fluctuation) and then consolidated to some number which will then correspond to a Letter (ASCII value of the letter). Those consolidated number and ASCII value of the letter are stored in a look-up table. Have a look at the code. I think it will be self-explanatory. Get me back for any further clarifications!

Share this post


Link to post
Share on other sites

Do you have a video of the device in action?

 

How does speed compare with typing?

 

Are you only doing fingerspelling, or are you recognizing word-signs as well?

Speed now is very slow when compared to typing. But can be improved in future. Currently only fingerspelling is supported. But same can be extended to word-signs as well. When word-signs are introduced, I strongly believe the speed will be better than typing. Currently I don't have any video. Get back to me if there are any more queries...

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×