Saturday, December 30, 2006

Gesture Recognition using Webcam and Laser Pointer

Controlling Windows Media Player:




I love listening to music on my laptop, but changing tracks is a real pain since I have to reach for the keyboard every time I feel like changing a song (and also because I'm very lazy)! Well, as the old saying goes, "laziness is the seed from which inventions flower". Okay, even though I just made that up, it's no less true! Anyway, I thought that it would be great if I could somehow give visual “gesture” commands to my laptop and control Windows Media Player without having to touch the keyboard. So, I fired up my code editor and wrote a laser gesture recognition program which currently recognizes up to eight visual gestures (left, right, up, down, 2 downward diagonals and 2 upward diagonals). Check out the videos above. One video shows a webcam view from the program and the other one shows how I’m controlling Windows Media Player with a laser beam.

Well, this isn’t the first time I created a laser control system. A few months back, I wrote a program which allowed me to create hotspots inside my house which could be activated with a laser beam. Each hotspot was programmed to take a pre-defined action. However, the problem with creating hotspots is that you have to remember where they are. If your room’s dark, you’ll have a really hard time finding and activating them. However, with gesture recognition, all you have to do is make a gesture anywhere in the camera’s field of view and that’s it! Well, here's a screenshot of my old program:


What next? Well, if I ever manage to hook up things in my house with my computer, I would love to create a virtual, laser controlled switchboard sorta thing to control them! Now, who needs a remote control when you can control everything with a laser pointer??

I’d love to hear your wonderful ideas and suggestions.

Keep visiting and have a HAPPY NEW YEAR!

Share/Bookmark

49 comments:

kernel_geek said...

Wow that sound amazing. Could you send me a copy of the compiled app ? Do i need a specific webcam? If you can pull yourself from your laser-pointer please email me at heelfliper@gmail.com. Thanks :D

NullReference said...

Very nice work! I am amazed at the work you are doing.

I am especialy interested in the C# code to grab frams from the webcam. I hope you will release some source for us to look at.

Thanks and keep having fun.

David Fowler said...

Great article! I had no idea that it would be so easy to control the stepper motor in a floppy drive.

I would like to see the source code.

I am also doing some interesting projects at my site/blog and the floppy drive idea could come in handy.

Ashish Derhgawen said...

I'll release the source code of my gesture recognition stuff soon.

David, I'm glad you enjoyed the floppy thing. I've release the source in a CodeProject article (http://www.codeproject.com).

:)

Anonymous said...

Great project, can't wait to get my hands on it.

Peter said...

Looking good,

waiting for the release of the code as well :o)

Anonymous said...

Looks like this guy did it earlier and faster:

http://codeninja.de/laserpointergestures/

Anonymous said...

Anonymous said...

Looks like this guy did it earlier and faster:
http://codeninja.de/laserpointergestures/

Yes, but he doesn't show any intention to release any code...

Anonymous said...

any info on where and when you will make this code available?

Doesn't have to be the cleanest code in the world :)

Ashish Derhgawen said...

http://www.codeproject.com/useritems/lasergesture.asp

:)

Ed said...

Great idea, can't wait to try it out, but I downloaded the demo software 'motion' and it says application error (0xc0000135) - don't suppose you could help me out?

Thanks

Ashish Derhgawen said...

You need to have the .NET Framework installed. Get .NET 2.0: http://www.microsoft.com/downloads/details.aspx?familyid=0856eacb-4362-4b0d-8edd-aab15c5e04f5&displaylang=en

Ashish

Matt said...

Hi Ashish,

What camera do you use? To me it seems like you'd need to worry about blurring.

Matt

Ashish Derhgawen said...

Hey Matt,

Which camera are you talking about? If you're talking about the camera I used for recording the video at the top of this post..its a Sony Cybershot. If you're talking about my webcam..its a really cheap camera. :) Well, I don't really care much about blurring because my gesture recognizer doesn't really worry about how sharp the video is. :)

Alex said...

Hey!, just wondering if u could also send me the Hotspot program u created
to dapaperclip@gmail.com

It would very much be appreciated. and if theres anything I could do I would be more than glad to help.

Alex said...

OHHH THATS PIMKPIEN

Eddie said...

Whoa, this is really incredibly cool! I was wondering, though, if there was any way to make this control the "Zune" program instead of WMP. They're both pretty much the same exact program, so is there a way just to substitute a name or something to make it work for Zune?

Thanks, and once again, this is incredible!

Ashish Derhgawen said...

I guess. You could try modifying the code to see if it can work with Zune: http://www.codeproject.com/useritems/lasergesture.asp

Let me know if you're successful. :)

Ben said...

Thank's Ashish. I just got your program set up today and already modified it. I set up an option to control winamp as well as windows media player and another option to control the mouse with the laser pointer. If you want my mods let me know

Ashish Derhgawen said...

Great job Ben! I had also modified my program to control the mouse. I'd love to see how your works. Could you email it to me? You'll find my email address under the "Contact Me" section of this blog.

Anonymous said...

2 Ashish: really awesome!
2 Ben: Can u send me a compiled exe of winamp controller? I can not modify the source because my destiny is Delfi :)
I will be very glad if you ll do that: galahed@ultra.ru
Sorry for English, from Russia with love, Galahed.

macatak said...

What would be really interesting would be if you could get a camera to pick up some kind of invisible light (UV or something). Anyone knwo of such a laser pointer?

Shashank said...

Hey Ashish this is Shashank form Amritsar. Man i am trying same sort of stuff but instead of controlling window media player i am trying to control some sort of devices using parallel port. Man this is for my final project and i need help urgently. Can u tell me any way to make frame grabber in vb.net.

Ashish Derhgawen said...

Hey Shashank,

I suppose this is what you wanna do... http://ashishrd.blogspot.com/2007/01/parallel-port-relay-for-playing-with.html

...right? I wrote the code in C#. You could try searching for VB code using DirectShow on CodeProject (http://www.codeproject.com)

Chris Clark said...

Hey Ashish.
i was wondering if you could also give the code for the hotspot program? i found similar code but it's all for linux. thanks a lot. my e-mail is boozelclark@gmail.com .
thanks again.

Rick said...

Love your work.

I am trying to follow a laser dot with precision as it moves across a target.
Could you please send me a copy of any code that will enable me to data map the points of the movement.
Thanks in adavance.

Ashish Derhgawen said...

@rick - Check this out: http://www.codeproject.com/cs/media/lasergesture.asp

:)

mitchell-o-t said...

Amazing, but doesn't it seem like more work doing all this than getting up and walking to your laptop?

Anonymous said...

it is really Amazing work
but can you send the code plz to my email or the program

plz to tupoleev@gmail.com

and thnx very

Anonymous said...

Hi Ashish,

I need to get in touch with you. Can you please give me your email ID?

I am leasturgent@yahoo.com working on a project www.skytran.net

Anonymous said...

great program, but is there a way to disable the beeping when it recognizes a command?

TBM said...

Any chance of e-mailing the hotspot program to rich200313@gmail.com? Thanks in advance.

Vandana said...

Hey what you did is great. Is there any chance you could develop a sign language recognition system using webcam? Could you please e-mail me your code to peridotgem@gmail.com?
Thank you

Real Vision said...

Good Stuff this!
I think you have a creative streak in you Ashish, and backed up with your programming prowress its a killer combo.

I was looking for a non conventional method to control a laptop running MediaPortal,that i decided should be my media center.

Well as i dont know programming I'll try and find something, but keep up the great work!
Regards!

Charles said...

Great Work!

Was just looking through your code and was wondering....u mentioned u used Andrew Kirilov's Motion Detection code for image acqusition.

Did you employ the motion detection part to detect if there is any motion of the laser or u just used the code for image acquisition? Coz i noticed you employed filters in the processing filter sequence but u did not use the images after filtering....

Ashish Derhgawen said...

Hey Charles: Yes, I used Andrew Kirilov's code for image acquisition. However, didn't use it for motion detection. Yes, there are filters in the filter sequence, but I didn't use them either.

Ruben said...

Hi Ashish, I'm trying to modify your code so I can control my winamp in stead of WMP, but I did not yet succeed in changing 'mediaplayerhandle=findwindow()'
Can you help me out with that? Great program, thanks!

munchman said...

Could you please email me the hotspot proram (mm.techtips@gmail.com), or put up a download link. That would be greatly apreciated. Thanks!

Anonymous said...

Hi Ashish!
Your program is great.
But I want to add keys like volume up and down, so I add it:
case "DIAGONAL1A":
SetForegroundWindow(mediaPlayerHandle);
SendKeys.SendWait("F9");
break;
case "DIAGONAL1B":
SetForegroundWindow(mediaPlayerHandle);
SendKeys.SendWait("F8");
break;

And it's not working, It's just passing to the media player window.
When I click F8 or F9 by myself it's changing the volume.

Ashish Derhgawen said...

@Anonymous:
Function keys are encoded in a string enclosed by {braces}.

This will not work:
SendKeys.SendWait("F9");

This should work:
SendKeys.SendWait("{F9}");

Cheers!

tukker64 said...

you should set up a beamer and a webcam and make software that controlls the cursor with the laser :P

Anonymous said...

did you use neural networks for the gesture recognition? or another technique? i am writing a mini paper as part of my degree and am trying to fine examples.

thank you,

Lee

Clapon said...

Hello, i would like to do such things, but i don't know how to start. I have some experience with image recognition (more specific with captcha braking / recognition) with PHP, i know real time image recognition (via a webcam) can't be done with PHP.
What programming language are you using? can you recommend some articles or tutorials to start with and to create a basic view over the real time computer vision.

Navshiv said...

Hey Ashish, I'm really inspired from this post of yours..and speacially idea of controlling WMP..well I've further extended your work..with more powerful gesture recognision without using neural network...and I've made another intresting program to control WMP using motion..you can check details here..
http://shivamkalra.blogspot.com

Im happy you are back after so long..keep up the good work!! YOU ROX..

Shivam

Rado R. said...

I have the feeling that you reinvent the wheel, man.
This dude here http://codeninja.de/laserpointerremote/ did it a long time ago, and then he did this: http://codeninja.de/laserpointergestures/ which is far advanced and lightweight you can see how fast the software responds on gestures, which are more complex than yours and still a (far) better response time (in case you haven't seen it from him already, anyway).

Anonymous said...

hi my name is melwin jose..(1st yr computer science)

can u send me the code of "Gesture Recognition using Webcam and Laser Pointer"...if it is in turbo c++...

if u have any other interesting project codes in c++ plz do send

my id: maxx913@gmail.com

ajay said...

hi
am lt ajay and am working on your project since last one month, but could not break through.can you please guide me to go through the project.please help me as am running short of time.please reply soon
thank

Kapil Bodke said...

Hi Ashish, i did saw your article and the efforts that you have taken. i did also checked the link that was published by you but failed to get the code. if possible can you please mail me the code and the system requirements at kapilbodke@gmail.com . i hav a basic laptop with inbuilt webcam hope this can work.

Aditi Chourasia said...

hii ashish your project and effort u took is great.. i have checked the link u have provided for detailed information.but i failed to get complete info..can you please mail it to adi2415@gmail.com asap..i need it as early as possible..