I need to find a good way to locate objects in the game, later I will find the coordinates of the object and send it to the Arduino controlling the stepper motors.
Step one is to install OpenCV on my Macbook. The best tutorial you will find on PyImageSearch.
First thing I tried was OpenCV with template matching. This is a method for searching and finding the location of a template image in a larger image. It simply slides the template image over the input image and compares the template and patch of input image under the template image. The method worked fairly OK, but it seemed like the method world be best on actual screengrabs, not motion video which can have a skew angle and differ in level of zoom. The template and object you want to find should be the same size.
This is when I discovered Haar Feature-based Cascade Classifier for Object Detection. This seems like the best algorithm to use for my case.
I have cloned a good tutorial for learn how to train your own OpenCV Haar classifier, which I used in combination with this tutorial. All of the classifiers I have trained, will be uploaded here.
Pyton code for realtime testing of multiple classifiers:
# Original code from http://www.trevorsherrard.com/Haar_training.html
# Code is modfied by Karstein Kvistad - www.phobic.no - March 2017
# Opens up a webcam feed and tests multiple classifier in real time using detectMultiScale
def detect(img, name):
cascade = cv2.CascadeClassifier(name)
rects = cascade.detectMultiScale(img, 1.3, 4, cv2.cv.CV_HAAR_SCALE_IMAGE, (20,20))
if len(rects) == 0:
return , img
rects[:, 2:] += rects[:, :2]
return rects, img
# r,g,b input for tracking box color
def box(rects, img, r, g, b):
for x1, y1, x2, y2 in rects:
cv2.rectangle(img, (x1, y1), (x2, y2), (r, g, b), 2)
cap = cv2.VideoCapture(0)
ret, img = cap.read()
rects, img = detect(img, "cascade_green_arrow.xml")
box(rects, img, 127, 255, 0)
rects, img = detect(img, "cascade_donut.xml")
box(rects, img, 255, 105, 180)
if(cv2.waitKey(1) & 0xFF == ord('q')):
# do a bit of cleanup
print("[INFO] cleaning up...")
EDIT: I discovered a fault that sent me on an hour detour around Google. At the step:
perl bin/createsamples.pl positives.txt negatives.txt samples 1500 "/usr/local/Cellar/opencv/2.4.12/bin/opencv_createsamples -bgcolor 0 -bgthresh 0 -maxxangle 1.1\ -maxyangle 1.1 maxzangle 0.5 -maxidev 40 -w 201 -h 60"
I only get the error:
Illegal division by zero at bin/createsamples.pl line 45.
I tried different things, like changing location of positive, negative and samples folder, adding the whole file location for /usr/local/Cellar/opencv/2.4.12/bin/opencv_createsamples and messing around with the opencv_createsamples parameters. Nothing helped.
At some comment field I found the solution. I had only used one positive file when using the command:
find positive_images -iname "*.png" > positives.txt
This resulted in a positives.txt file having only one line of text. The createtrainsamples.pl ignores the last entry and throws an exception.
Just adding an empty line fixed the problem, and the script finally works!
<em>Karsteins-MacBook-Air:</em>opencv-haar-classifier-training phobic$ perl bin/createsamples.pl positives.txt negatives.txt samples 1500 "/usr/local/Cellar/opencv/2.4.12/bin/opencv_createsamples -bgcolor 0 -bgthresh 0 -maxxangle 1.1\ -maxyangle 1.1 maxzangle 0.5 -maxidev 40 -w 201 -h 60"
/usr/local/Cellar/opencv/2.4.12/bin/opencv_createsamples -bgcolor 0 -bgthresh 0 -maxxangle 1.1\ -maxyangle 1.1 maxzangle 0.5 -maxidev 40 -w 201 -h 60 -img positive_images/collect.png -bg tmp -vec samplesositive_images/collect.png.vec -num 1500
Info file name: (NULL)
Img file name: positive_images/collect.png
Vec file name: samplesositive_images/collect.png.vec
BG file name: tmp
BG color: 0
BG threshold: 0
Max intensity deviation: 40
Max x angle: 1.1
Max y angle: 1.1
Max z angle: 0.5
Show samples: FALSE
Original image will be scaled to:
Width: $backgroundWidth / 201
Height: $backgroundHeight / 60
Create training samples from single image applying distortions...