Gnome Voice Control – Demonstration

16 06 2007

Hi, I made a brief demonstration’s video to show the functionalities of the Gnome Voice Control First Release that is been developed on GSoC.

I hope that you enjoy.

I would like to thank luizirber more known as Gaucho, by the help on making the demonstration video.

The software can be found on:

http://www.comp.ufscar.br/~raphael18/gnome-voice-control-0.1.tar.gz

Instructions to instalation can be found on:

http://raphaelnunes.wordpress.com/2007/06/07/gnome-voice-control-01-release-gsoc/ 

Thanks,
Raphael Nunes

About these ads

Actions

Information

32 responses

16 06 2007
mike

Can you also say: “Show me Mail/Evolution?”

16 06 2007
Marcos

oh wow…this is so cool! very very awesome work man! cant wait for an official final release :) keep up the admirable work!

16 06 2007
nacho

very cool indeed!

16 06 2007
Ubuwu

Looks promising!

16 06 2007
Og Maciel

Hey Raphael, great work! I downloaded your code and tried to take it for a spin, but got hung up on a few compilation errors. Am I missing a dependency (based on the message below)?

Cheers,

Og

In file included from voice-control.c:33:
gstsphinxsink.h:30:21: s2types.h: Arquivo ou diretório não encontrado
gstsphinxsink.h:31:17: fbs.h: Arquivo ou diretório não encontrado
gstsphinxsink.h:32:23: CM_macros.h: Arquivo ou diretório não encontrado
gstsphinxsink.h:33:16: ad.h: Arquivo ou diretório não encontrado
gstsphinxsink.h:34:21: cont_ad.h: Arquivo ou diretório não encontrado
In file included from voice-control.c:33:
gstsphinxsink.h:57: error: parse error before “int32″
gstsphinxsink.h:57: warning: no semicolon at end of struct or union
gstsphinxsink.h:58: warning: data definition has no type or storage class
gstsphinxsink.h:59: error: parse error before “bps”
gstsphinxsink.h:59: warning: data definition has no type or storage class
gstsphinxsink.h:60: error: parse error before “calibrated”
gstsphinxsink.h:60: warning: data definition has no type or storage class
gstsphinxsink.h:61: error: parse error before “initialized”
gstsphinxsink.h:61: warning: data definition has no type or storage class
gstsphinxsink.h:62: error: parse error before “calibrate_started”
gstsphinxsink.h:62: warning: data definition has no type or storage class
gstsphinxsink.h:63: warning: data definition has no type or storage class
gstsphinxsink.h:73: error: parse error before “cont_ad_t”
gstsphinxsink.h:73: warning: no semicolon at end of struct or union
gstsphinxsink.h:74: warning: data definition has no type or storage class
gstsphinxsink.h:82: error: parse error before ‘}’ token
make[2]: ** [voice-control.o] Erro 1
make[2]: Leaving directory `/home/omaciel/conary/builds/gnomevoicecontrol/gnome-voice-control-0.1/src’
make[1]: ** [all-recursive] Erro 1
make[1]: Leaving directory `/home/omaciel/conary/builds/gnomevoicecontrol/gnome-voice-control-0.1′
make: ** [all] Erro 2
error: Shell command “cd ‘/home/omaciel/conary/builds/gnomevoicecontrol/gnome-voice-control-0.1/’; CFLAGS=”-O2 -march=i686 -g” CXXFLAGS=”-O2 -march=i686 -g ” CPPFLAGS=”” CLASSPATH=”” LDFLAGS=”-g” CC=gcc CXX=g++ make ” exited with exit code 2
/usr/lib/python2.4/site-packages/conary/build/packagerecipe.py:1014: RuntimeError: Shell command “cd ‘/home/omaciel/conary/builds/gnomevoicecontrol/gnome-voice-control-0.1/’; CFLAGS=”-O2 -march=i686 -g” CXXFLAGS=”-O2 -march=i686 -g ” CPPFLAGS=”” CLASSPATH=”” LDFLAGS=”-g” CC=gcc CXX=g++ make ” exited with exit code 2

16 06 2007
bersace

Is i18n planned ?

16 06 2007
bersace

I forgot to say : you rock ! :)

16 06 2007
bandy

HI
it’s really cool (the demo:))
i installed it on openSUSE Factory, but nothing works, how can i “debug” it, or something? it compiles OK

16 06 2007
Mike

Very good project! This will help Linux be so much more accessible, as well as take the conventional user interface to a new level. The demo only showed run, minimize, maximize, next, and close. While these are great starting points (and I do realize this is very much a work in progress), is there other functionality, like run x, where x can be any program of your choice? Can it access menu items in applications?

Looking forward to see how this develops! :) Great work, guys.

16 06 2007
nsh

> gstsphinxsink.h:30:21: s2types.h: Arquivo ou diretório não encontrado
> gstsphinxsink.h:31:17: fbs.h: Arquivo ou diretório não encontrado

You have to install sphinx2 first and correctly point it’s location to configure

> i installed it on openSUSE Factory, but nothing works, how can i “debug” it, or
> something? it compiles OK

Does sound recording with gstreamer works for you? You should be able to test or setup it in Preferences/Multimedia System capplet

16 06 2007
nsh

> Can you also say: “Show me Mail/Evolution?”

Not now, but of course we can add it

> Is i18n planned ?

Yes. For now there are French, Russian and Spanish models. The rest depends on availability of free speech data.

16 06 2007
Top Posts « WordPress.com

[...] Gnome Voice Control – Demonstration Hi, I made a brief demonstration’s video to show the functionalities of the Gnome Voice Control First Release […] [...]

17 06 2007
Lucas Rocha

Raphael, very nice! Keep up with your excelent work! :-)

17 06 2007
Sander

For me it hangs at calibration too.

17 06 2007
Joel

Hello,

Wanted to give it a go, but http://www.comp.ufscar.br seems to be down from my end. Is the code hosted anywhere else?

17 06 2007
Alfonso

Awesome! But we also want a version for KDE :_(
Keep up the good work :))

17 06 2007
nsh

> Wanted to give it a go, but http://www.comp.ufscar.br seems to be down from my end. Is the code hosted anywhere else?

You can check it out from berlios svn repository:

svn checkout http://svn.berlios.de/svnroot/repos/festlang/trunk/gnome-voice-control

> For me it hangs at calibration too.

It means you have no sound recorded through gstreamer. Please adjust your mixer settings.

17 06 2007
Sander

I’ve set-up my microphone in gstreamer and the volume is high on both recording and playback, it still hangs though..

18 06 2007
nsh

Hm, rather strange. Does the following pipeline record anything?

gst-launch gconfaudiosrc ! audioconvert ! vorbisenc ! oggmux ! filesink location=test.ogg

18 06 2007
Sander

Yeah, I’m getting a nice recording of me singing ;-)

19 06 2007
nsh

another hm, can you please collect a backtrace then:

ps x| grep voice_control_applet
gdb –pid
thread apply all bt

I think it’s better to move this discussion to mail though. Can you please write Raphael and CC me (nshmyrev@yandex.ru).

26 06 2007
3 07 2007
Hubby

I watched the first and last seconds how you are opening and closing windows. If the rest ist as boring as what I saw, I wonder why you are wasting so much of your and our time?

22 12 2007
speech recognition software

Your demonstration is very interesting. There is a lot of work to be done to develop a good free and open source speech recognition software. But your demonstration shows that the first steps are done.

26 12 2007
Vivek

Mine also hangs at calibration..using ALSA mixer all at full volume no mutes…

3 01 2008
Luis Medina

I’m very interested on voice control on Linux and Speach Recognition to dictare on linux, spacificly on Ubuntu. This work shows that it’s posible but how much has been done. Some recomendation?

23 02 2008
Lumenos

If you can make it simulate keyboard strokes you can do all kinds of things.

If you can get it to type numbers 1-9 you could enable surfing the web with Firefox, with the Mouseless Browsing Add-on. There is an option so you don’t have to press “Enter”; you say the numbers and it goes. In Windows I use it with Dragon NaturallySpeaking but it jumps around on the page for some reason. I don’t think your software would do that.

If you can make it press multiple keys at once, you can do keyboard shortcuts. There is probably software that can do all kinds of macros from keyboard shortcuts. On Gnome you can use ALT + TAB or ALT plus F1 through F4 to get to the windows and menus, or run programs. If you can make it follow a command like “press Down Arrow five times” you should be able to quickly select files with a file manager. (Press the first letter of what you want to select in the file manager then press the Down Arrow five times.)

If you can make it hold a key down, you could use ALT + TAB for selecting windows or CTRL + TAB for selecting tabs in Firefox.

If you say “mouse” followed by a number, Dragon NaturallySpeaking divides the screen into nine sectors, numbers all the sectors, and puts the pointer in the sector whose number you chose. It divides this sector into nine more sectors and so on. If you have a fast enough computer this can be as fast as moving a mouse input device. Gnome may already have some way of doing this with the keyboard.

I’ve been trying to get WinDictator to work, here’s the article in my wiki for that.
http://lumeniki.scribblewiki.com/Voice_control_and_speech_recognition_on_free_operating_systems

However all I really need WinDictator for, is keyboard shortcuts. If your software could be used for the keyboard shortcuts (voice control), Windows/DNS in a virtual machine, could be used for writing words (speech recognition). You can cut and paste text with VMware. (I can record audio on Ubuntu while DNS is transcribing in Windows in the virtual machine, but when I went to check the DNS speech-to-noise test, Ubuntu popped up a message saying the audio device is not available. That is probably because I had the virtual machine window in focus. It didn’t stop transcription.) Computers with dual and quad core processors are affordable now. Running DNS on Windows alone doesn’t seem to be any faster than running it in the virtual machine on my Core 2 Quad with 3 GB of RAM (two GB and “two CPUs” for the virtual machine) and a good mic.

26 06 2008
twig

It might work better if he didn’t have such a thick accent :)

31 07 2008
hamen

Hi, I followed your instructions. I get this error when I run “make”:

In file included from gstsphinxsink.c:33:
gstsphinxsink.h:32:17: error: fbs.h: No such file or directory
gstsphinxsink.c:321: error: expected declaration specifiers or ‘…’ before ‘s2_fsg_trans_t’
gstsphinxsink.c: In function ‘gst_sphinx_construct_trans_list’:
gstsphinxsink.c:325: error: ‘s2_fsg_trans_t’ undeclared (first use in this function)
gstsphinxsink.c:325: error: (Each undeclared identifier is reported only once
gstsphinxsink.c:325: error: for each function it appears in.)
gstsphinxsink.c:325: error: ‘transitions’ undeclared (first use in this function)
gstsphinxsink.c:337: error: expected expression before ‘)’ token
gstsphinxsink.c:360: error: ‘trans_list’ undeclared (first use in this function)
gstsphinxsink.c: In function ‘gst_sphinx_sink_set_fsg’:
gstsphinxsink.c:371: error: ‘s2_fsg_t’ undeclared (first use in this function)
gstsphinxsink.c:371: error: expected ‘;’ before ‘fsg’
gstsphinxsink.c:372: error: ‘s2_fsg_trans_t’ undeclared (first use in this function)
gstsphinxsink.c:372: error: ‘trans_list’ undeclared (first use in this function)
gstsphinxsink.c:374: error: ‘fsg’ undeclared (first use in this function)
gstsphinxsink.c:375: error: too many arguments to function ‘gst_sphinx_construct_trans_list’
make[2]: *** [gstsphinxsink.o] Error 1
make[2]: Leaving directory `/home/ivan/temp/gnome-voice-control-0.3/src’
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/ivan/temp/gnome-voice-control-0.3′
make: *** [all] Error 2
In file included from gstsphinxsink.c:33:
gstsphinxsink.h:32:17: error: fbs.h: No such file or directory
gstsphinxsink.c:321: error: expected declaration specifiers or ‘…’ before ‘s2_fsg_trans_t’
gstsphinxsink.c: In function ‘gst_sphinx_construct_trans_list’:
gstsphinxsink.c:325: error: ‘s2_fsg_trans_t’ undeclared (first use in this function)
gstsphinxsink.c:325: error: (Each undeclared identifier is reported only once
gstsphinxsink.c:325: error: for each function it appears in.)
gstsphinxsink.c:325: error: ‘transitions’ undeclared (first use in this function)
gstsphinxsink.c:337: error: expected expression before ‘)’ token
gstsphinxsink.c:360: error: ‘trans_list’ undeclared (first use in this function)
gstsphinxsink.c: In function ‘gst_sphinx_sink_set_fsg’:
gstsphinxsink.c:371: error: ‘s2_fsg_t’ undeclared (first use in this function)
gstsphinxsink.c:371: error: expected ‘;’ before ‘fsg’
gstsphinxsink.c:372: error: ‘s2_fsg_trans_t’ undeclared (first use in this function)
gstsphinxsink.c:372: error: ‘trans_list’ undeclared (first use in this function)
gstsphinxsink.c:374: error: ‘fsg’ undeclared (first use in this function)
gstsphinxsink.c:375: error: too many arguments to function ‘gst_sphinx_construct_trans_list’
make[2]: *** [gstsphinxsink.o] Error 1
make[2]: Leaving directory `/home/ivan/temp/gnome-voice-control-0.3/src’
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/ivan/temp/gnome-voice-control-0.3′
make: *** [all] Error 2

I installed ubuntu deb, but it has a known bug and i does not work. I decided to compile from sources. Here I am… Any help? Tnx

29 03 2009
RaiulBaztepo

Hello!
Very Interesting post! Thank you for such interesting resource!
PS: Sorry for my bad english, I’v just started to learn this language ;)
See you!
Your, Raiul Baztepo

23 04 2009
Jefferythewind

Hi, i have just tried installing all of this. I compiled the sphinx things, no problem. But i am getting the same problem as above:

gstsphinxsink.h:32:17: error: fbs.h: No such file or directory
gstsphinxsink.c:321: error: expected declaration specifiers or ‘…’ before ‘s2_fsg_trans_t’
gstsphinxsink.c: In function ‘gst_sphinx_construct_trans_list’:
gstsphinxsink.c:325: error: ‘s2_fsg_trans_t’ undeclared (first use in this function)
gstsphinxsink.c:325: error: (Each undeclared identifier is reported only once
gstsphinxsink.c:325: error: for each function it appears in.)
gstsphinxsink.c:325: error: ‘transitions’ undeclared (first use in this function)
gstsphinxsink.c:337: error: expected expression before ‘)’ token
gstsphinxsink.c:360: error: ‘trans_list’ undeclared (first use in this function)
gstsphinxsink.c: In function ‘gst_sphinx_sink_set_fsg’:
gstsphinxsink.c:371: error: ‘s2_fsg_t’ undeclared (first use in this function)
gstsphinxsink.c:371: error: expected ‘;’ before ‘fsg’
gstsphinxsink.c:372: error: ‘s2_fsg_trans_t’ undeclared (first use in this function)
gstsphinxsink.c:372: error: ‘trans_list’ undeclared (first use in this function)
gstsphinxsink.c:374: error: ‘fsg’ undeclared (first use in this function)
gstsphinxsink.c:375: error: too many arguments to function ‘gst_sphinx_construct_trans_list’
make[2]: *** [gstsphinxsink.o] Error 1
make[2]: Leaving directory `/home/jeffery/gnomevoicecontrol/gnome-voice-control-0.3/src’
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/jeffery/gnomevoicecontrol/gnome-voice-control-0.3′
make: *** [all] Error 2

I would love to get this gong on my system, any help would be appreciated.

12 06 2009
guru

i’m having the same exact error on Ubuntu 9.04

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s




Follow

Get every new post delivered to your Inbox.

%d bloggers like this: