ASL Finger spelling recognition using C# OpenCV wrapper
Vložit
- čas přidán 23. 01. 2010
- ASL Finger spelling recognition. 19 identified signs, (currently still working on 7 other signs).
Github repo: github.com/Juniar-Rakhman/Ope...
Translasi bahasa isyarat mengklasifikasi 19 isyarat huruf tangan 7 isyarat lainnya masih belum bisa.
- Haar object detection (detect open and closed hand) to determine the position of the box.
- Movement detection (absolute difference). If shaking then reset ROI box, if no movement then detect the shape.
- To extract the hand shape I use skin detection based on HSV or RGB (it depends on the lighting) then convert it to binary image.
- To classify the image I use KNN (look up the machine learning section of OpenCV) I use 100 training images per signs.
paper : www.eepis-its.edu/uploadta/dow... (sorry it's in Indonesian)
source code:
www.4shared.com/file/2AwWOJ_V/...
Huge credit goes to Heiko Kiessling for creating easy to use C# OpenCV wrapper and helping me (alot) in creating this program.
You can find his code project article here :
www.codeproject.com/Articles/2....
Noob guide:
1. Download opencv from here :
sourceforge.net/projects/openc...
(I used the old version of opencv 1.1pre something)
2. Download OpenCV C# wrapper from here :
www.iib-chemnitz.de/cvwrapper/...
(I used the one for opencv 1.1 pre)
3. Make sure that you can successfully run the big sample from the wrapper before attempting to compile/run my program.
4. You may have to edit and play around with the code because some settings such as resolution is hardcoded. May the code be with you.
some rights reserved :) - Věda a technologie
I was just thinking about giving this a shot myself but then searched and saw you're working on it! Good job!
you have such good programming skills 10 year back and I am learning this in 2020
wahh mantap nih...
coba pake gesture and finger recognition, pasti cakep
@ank3201
First of all make sure that the haar training detection works. If it doesn't maybe you should consider making your own haar training. The haar cascade I used works well in well-lighted roomt. If the haar detection works, then the hand shape classification should work as well.
Please note that I used 640x480 resolution with Logitech Quickcam 9000.
Really cool!!
wkwkwk suwun2 mw sidang dosene ga ono sing teko macem2 ^_^ soale presentasi boso inggrisan
keren tuh bang,,,,bisa buat referensi nih :)
hi, in code xml do ASL? for description?
how to start program? Error: An unhandled exception of type 'System.NullReferenceException' occurred in cvlibcs.dll
I'm still working on 7 other hand signs and some minor improvements. I'll post another video when it's done ^^
also, which haar classifiers did you use to detect the hand (ie object): horizontal or vertical etc?
Hi, you have a cam Shift example in EmguCV C#?
Thanks
why is that with the cvCreateCameraCapture(0) it sometimes failed and sometimes it runs but it is black screen? :((
Hey @nakagawa53. Have couple of questions. i) Does it detect hand when u bring the hand ahead of the face ? ii) Does using Haar clasifiers slow down the program ??
would you please share the training images for each signs? that should be in Training folder? Thnx
i also need these images
I've updated the info section ^^
Webcam enough to recognition or need with it some software
can you please provide training images you used ??
@crsdaryl20 I'm just using thresholding to extract the shape. That's why the program hasn't been able to recognize similar signs to A (i.e S, E, M, N, and T) I tried to use canny...but the result is not very accurate.
i have some questions to you.
1. your ROI doesn't move it, why and how ?
@sanjeven sorry for the late reply I've been busy with my work lately, I've update the info section, there's new link to the complete source code and training files.
@nakagawa53 libraries such as? have you completed the 7 other signs left? thanks!
memperingati wisuda sing k 100
IT d kongkong boso inggrisan karo pak kholid ^^
Hi I'm trying to create something very similar to you for my final year project but for Irish sign language. However I can't seem to wrap my head around the instructions that you give as I'm very new to c#. Do I need to download anything for c# to work in textpad?
Any help would be greatly appreciated.
Does anyone have the code in Python?
it seems the you have a predefined area to get the hand image. is it possible to show another video of another area where you get the hand signs? maybe lower or the other hand. thanks... very good still
hello. i was wondering if i can use your software as reference for my thesis project. is it ok with you?
btw, are there any updates on this project? you stated before that it can identify 19 signs and that you're still working on the other 7. thanks a lot for your help.
bro ..how to you remove your arm and face if your follw skin detection
badalah.... keren amat...
most of the links posted are dead.... Please reupload on different location
how i can connect between c# project and opencv 1.1
Hey hi, I was just wondering, but what does the 'K' signify in the upper right corner???
what edge detection algo did you use?
@nakagawa53 tried running your program but got this error --> src_image = highgui.CvLoadImage(file, highgui.CV_LOAD_IMAGE_GRAYSCALE);
I am getting the same problem.
Will you please tell me that did you get the solution.
the training has been successful but I only have black screen. pls help
Why is it necessary to train the system each time you run the code?
ei how do u compare the hand sign from the database
@LittlePokeDigi hm...it's easy enough I guess...u just have to press A when the binary image shows up then it will add the captured binary image to the A training folder or press B to add the captured image to B training folder. If you want to add more signs that involve movement like J or Z you need to add finger tips tracking, optical flow etc. I haven't got enough free time to continue this project but feel free to modify my code.
I'm sorry to bother you but can you elaborate more on how to use K-nn ? I read you codes and understand how you take your sampling, but how do you actually classify a new sign? Which part actually gets the user's input on what that is when training?
i cant install OpenCv,, i used opencv 1.1 in your link.....there is another way or what will i do?
how did you build the file for trainig for KNN?
can you help me build my own from my own pictures ?
@catubig3710 sorry for the late reply I've been busy with my work lately, I've update the info section, there's new link to the complete source code and training files.
when i debug the program why is it always black? it didn't show some picture? :((
What's your accuracy percentage?
Disculpa en c# me aparece un error Application.Run(new MainForm()); Se ha intentado cargar un programa con un formato incorrecto. (Excepción de HRESULT: 0x8007000B)
tuviste exito?
can you please help me out in finger detection and recognition?
@MrSimplesify Info section barusan saya update boss
@catubig3710 well I'm not much of a java person. I think for image processing it's best to do it in C++. Using webcam should be enough but u might also combine it with sensors to achieve better result.
sir can you teach me how did you train the system in detecting the gestures :))
I haven't got much time to work on this project lately, so I decide to share the source code (download links added)
feel free play around with it ^^
Sorry about my (very) late reply. There are many people asking the same question.
Please note that you cannot extract the program, compile it, and hoping that it would work right away =_=" there are several libraries that you need to install.
I have updated the info section I hope it helps.
i got an error in src_image = highgui.CvLoadImage(file, highgui.CV_LOAD_IMAGE_GRAYSCALE); the comment is 'Object reference not set to an instance of an object.,,,would you like to help me,if the variable not yet declared where to put the declaration because i got an error....thanks but i need your help
Thanks Brothers
help me!
edw.0919@gmail.com
@@edw0919 How i am can help you ?
i have a error !! I can not execute it :( , VS 2010 :( plz
Hey hi!
First of all, great work. But when I tried running your source code the video input was not displayed in the window created. I've tried with different camera resolutions and different web cameras as well but to no avail. Also, it's unable to open a video file and work on it.
Could you please guide me as to what to do?
Assalam u alaikum.. I am developing such feature in a web app as my final year project and need your help for implementing this feature.. Kindly help.. Plz plz
Got The Same error Did You Got The Solution..?
would u mind to post the link from where i can download your source code. the given link of 4shared is not working.
plz give me the link. it will help me a lot to do something with that.
thanks in advance.
although i suspect that your program is ether forgotten or completed, i would recommend installing visual c# from Microsoft. Currently the 2012 edt. is free!
Greeting from the US!
how to run your code,what tje process...when i run i got an error,,would you like to help me???,,,what the steps,,i already installed opencv C# wrapper and opencv 1.1. pre something.....
Aselam Aleykum there,, the source code link is not working ,,, will u update us with working link
Can i get uour source code?thx for your help
I need the training folder (Training Images + handOpen.xml). If anyone have then please share.. Thanks alot.
@baamchika Info section updated.
mas programnya ini menyebabkan out of memory g?
soalnya tak cobak di komputerku out of memory ketika memory di aplikasinya mencapai 400 mb an
q udah nyari2 referensi release semua mulai dari CvReleaseMemStorage, CvReleaseImage, CvClearSeq tapi tetep out of memory
kira2 mas juniar tahu solusinya?
kalo penyebabnya tak cobak2 ada di baris handOpen=.... pada file HaarClassifer.cs
Sir are u got ur answer , if yes plz share it i got the same problem
great man....
The lecturers must be speechless...
LOL
Hi, i am working with a similar project and i am student from Argentina UNLaM. Could you help us with your code or explain us how to do that? Thanks a lot.
where is the update?
we are getting an error saying Object reference not set to an instance of an object
for this line src_image=highgui.CvLoadImage(file, highgui.CV_LOAD_IMAGE_GRAYSCALE);
so plzz help solve this error
I need training images and handOpen.xml please....
hey...
great work.
i visited the link but was unable to run the code. it wasn't detecting my hand. can u give me some solution how i can go about doing it
selamat malam mas, saya berniat belajar computer vision. kebetulan saya tidak punya sumber, kebanyakan abu-abu dari internet. saya kemarin baru buat project computer vision (mouse gesture) pakai JavaCV. itu pun masih banyak bugs.
BIsa minta kontak ?>
sir how to show the binary image in the crop window?
??please solution ?
the source code link doesn't working.
helo,,have a problem in your codes when i run it,,it has error.... src_image = highgui.CvLoadImage(file, highgui.CV_LOAD_IMAGE_GRAYSCALE);.......what wrong with this,i dont know what to do????...can u give me some advice...,,,the comment create is "Object reference not set to an instance of an object."
se puede usar OpenCv con C# ? 😱😱
Kak ini pakai PCA bukan?
the program runs but its always black screen.. what is the problem of that sir?
same in my case sir,
thanks for the source code. I would like to extend the project and make it better. In case i decide to, I will contact you.
Hi do you have your work done?
Halo kak. Saya Arif dari Indonesia. Jika berkenan saya ingin diajarkan cara membuat aplikasi ini. Terimakasih
and also the closehand.xml.
lu orng indo bang?
Amigos alguien pudo correr el programa me podria ayudar... ??
que mensaje te ha salido?
I'm really confused i can't run it, why "I've got the window app but nothing is running"
help me
help me!
my webcam is attached to PC via USB
Please help me with this project
help me , please!
Would you please share the training folder?
the training folder not share, but you can make it with step traditional, show video "ASL Finger spelling recognition using C# OpenCV wrapper" and then crop/print screen it and paste in Microsoft word then crop signs A..Z after that copy to paint and save as to ".png" / ".jpg", the finally ".png" chage to ".pbm" and Save.
or
you can make foto in extention .pbm with application else paint.
After that :
Make Folder 0 - 19 and fill each folder with foto in extension ".pbm"
#IHaveTryItAndSucces
Thank u so much.
I have a one more question, where can i find handOpen.xml and handClose.xml and also when run the program
and click 'start video' but its always black screen. Do u know solution of this situation ?
you can install software "manycam" and try it, i have tried and success but the resolution just 320 x 240.
in the real folder, that you have download it's available.
But for the easier use it. You can download in this link :
drive.google.com/open?id=0B3nB_txibXjId1hYS1NKXzJqMm8
i have upload the training folder.
Can you solv black screen problem?(When you press start video button, program shows black screen, no video).
Hi! i am Aminur from Bangladesh. I want to make a project to recognize Bengali Sign Language. So plz help me. Firstly i have run your code but i need Step by step instruction how the train menu will work.
Best regards
Aminur
If Yes Plz Post It
can i do it with python ?
help me this code ... i need...
thank you
Hi. I'm trying to make it with python as well. Have you made any progress you would be willing to share?
Persephone Tsebelis i'm use intel galileo and python . Control the toy car by hand with webcam.
sir im new to this and i dont know anything and im having this kind of error what do you think i need to do. pls sir i really need to make this work for my final project. TIA sir
An unhandled exception of type 'System.BadImageFormatException' occurred in cvlibcs.dll
Additional information: An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B)
Cerberus.Edson Tan hey, have you fixed this problem yet? I am facing this problem now..
the error is same mine and it's a file not found exception
help me, I have a Problem, print a message "No process the imagen"
waw... keren bgt... saya juga lagi berusaha untuk hand detection in c# juga ^^.. btw saya sudah kerem message ke @nakagawa53 .. mohon di baca ya ^^.. hope a great response
sangar,.,.,.,.,.,TA opo TA iki,.,.ancen DEWA koncoq iki,.,.,
tp sayang backgroundne nguilani jeh,.,.,
kwkwkwkwkwkwk,.,.,hahahahahhaha,.,.,:))
Please help me thank you
Same Error Did You Got The Solution..?
Plz Reply Soon
Selamat malam mas
Ohh man!! You did it 10 years ago....😂🙄LOL!!...
People like me are thinking it is the trending project nowadays.
Hi!!
This is a great video.. sent you a youtube mail... please check..thanks!