Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Free Learning
Arrow right icon
OpenCV with Python Blueprints
OpenCV with Python Blueprints

OpenCV with Python Blueprints: Design and develop advanced computer vision projects using OpenCV with Python

Arrow left icon
Profile Icon Michael Beyeler (USD) Profile Icon Michael Beyeler
Arrow right icon
$24.99 $35.99
Full star icon Full star icon Full star icon Full star icon Half star icon 4.4 (10 Ratings)
eBook Oct 2015 230 pages 1st Edition
eBook
$24.99 $35.99
Paperback
$43.99
Subscription
Free Trial
Renews at $19.99p/m
Arrow left icon
Profile Icon Michael Beyeler (USD) Profile Icon Michael Beyeler
Arrow right icon
$24.99 $35.99
Full star icon Full star icon Full star icon Full star icon Half star icon 4.4 (10 Ratings)
eBook Oct 2015 230 pages 1st Edition
eBook
$24.99 $35.99
Paperback
$43.99
Subscription
Free Trial
Renews at $19.99p/m
eBook
$24.99 $35.99
Paperback
$43.99
Subscription
Free Trial
Renews at $19.99p/m

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
OR
Modal Close icon
Payment Processing...
tick Completed

Billing Address

Table of content icon View table of contents Preview book icon Preview Book

OpenCV with Python Blueprints

Chapter 2. Hand Gesture Recognition Using a Kinect Depth Sensor

The goal of this chapter is to develop an app that detects and tracks simple hand gestures in real time using the output of a depth sensor, such as that of a Microsoft Kinect 3D sensor or an Asus Xtion. The app will analyze each captured frame to perform the following tasks:

  • Hand region segmentation: The user's hand region will be extracted in each frame by analyzing the depth map output of the Kinect sensor, which is done by thresholding, applying some morphological operations, and finding connected components
  • Hand shape analysis: The shape of the segmented hand region will be analyzed by determining contours, convex hull, and convexity defects
  • Hand gesture recognition: The number of extended fingers will be determined based on the hand contour's convexity defects, and the gesture will be classified accordingly (with no extended fingers corresponding to a fist, and five extended fingers corresponding to an...

Planning the app

The final app will consist of the following modules and scripts:

  • gestures: A module that consists of an algorithm for recognizing hand gestures. We separate this algorithm from the rest of the application so that it can be used as a standalone module without the need for a GUI.
  • gestures.HandGestureRecognition: A class that implements the entire process flow of hand-gesture recognition. It accepts a single-channel depth image (acquired from the Kinect depth sensor) and returns an annotated RGB color image with an estimated number of extended fingers.
  • gui: A module that provides a wxPython GUI application to access the capture device and display the video feed. This is the same module that we used in the last chapter. In order to have it access the Kinect depth sensor instead of a generic camera, we will have to extend some of the base class functionality.
  • gui.BaseLayout: A generic layout from which more complicated layouts can be built.
  • chapter2: The main script for the chapter...

Setting up the app

Before we can get down to the nitty-gritty of our gesture recognition algorithm, we need to make sure that we can access the Kinect sensor and display a stream of depth frames in a simple GUI.

Accessing the Kinect 3D sensor

Accessing Microsoft Kinect from within OpenCV is not much different from accessing a computer's webcam or camera device. The easiest way to integrate a Kinect sensor with OpenCV is by using an OpenKinect module called freenect. For installation instructions, take a look at the preceding information box. The following code snippet grants access to the sensor using cv2.VideoCapture:

import cv2
import freenect


device = cv2.cv.CV_CAP_OPENNI
capture = cv2.VideoCapture(device)

On some platforms, the first call to cv2.VideoCapture fails to open a capture channel. In this case, we provide a workaround by opening the channel ourselves:

if not(capture.isOpened(device)):
    capture.open(device)

If you want to connect to your Asus Xtion, the device variable...

Tracking hand gestures in real time

Hand gestures are analyzed by the HandGestureRecognition class, especially by its recognize method. This class starts off with a few parameter initializations, which will be explained and used later:

class HandGestureRecognition:
    def __init__(self):
        # maximum depth deviation for a pixel to be considered # within range
        self.abs_depth_dev = 14

        # cut-off angle (deg): everything below this is a convexity 
        # point that belongs to two extended fingers
        self.thresh_deg = 80.0

The recognize method is where the real magic takes place. This method handles the entire process flow, from the raw grayscale image all the way to a recognized hand gesture. It implements the following procedure:

  1. It extracts the user's hand region by analyzing the depth map (img_gray) and returning a hand region mask (segment):
    def recognize(self, img_gray):
        segment = self._segment_arm(img_gray)
  2. It performs contour analysis on the hand region...

Hand region segmentation

The automatic detection of an arm, and later the hand region, could be designed to be arbitrarily complicated, maybe by combining information about the shape and color of an arm or hand. However, using a skin color as a determining feature to find hands in visual scenes might fail terribly in poor lighting conditions or when the user is wearing gloves. Instead, we choose to recognize the user's hand by its shape in the depth map. Allowing hands of all sorts to be present in any region of the image unnecessarily complicates the mission of the present chapter, so we make two simplifying assumptions:

  • We will instruct the user of our app to place their hand in front of the center of the screen, orienting their palm roughly parallel to the orientation of the Kinect sensor so that it is easier to identify the corresponding depth layer of the hand.
  • We will also instruct the user to sit roughly one to two meters away from the Kinect, and to slightly extend their arm...

Hand shape analysis

Now that we know (roughly) where the hand is located, we aim to learn something about its shape.

Determining the contour of the segmented hand region

The first step involves determining the contour of the segmented hand region. Luckily, OpenCV comes with a pre-canned version of such an algorithm—cv2.findContours. This function acts on a binary image and returns a set of points that are believed to be part of the contour. As there might be multiple contours present in the image, it is possible to retrieve an entire hierarchy of contours:

def _find_hull_defects(self, segment):
    contours, hierarchy = cv2.findContours(segment, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)

Furthermore, because we do not know which contour we are looking for, we have to make an assumption to clean up the contour result. Since it is possible that some small cavities are left over even after the morphological closing—but we are fairly certain that our mask contains only the segmented...

Hand gesture recognition

What remains to be done is to classify the hand gesture based on the number of extended fingers. For example, if we find five extended fingers, we assume the hand to be open, whereas no extended fingers implies a fist. All that we are trying to do is count from zero to five and make the app recognize the corresponding number of fingers.

This is actually trickier than it might seem at first. For example, people in Europe might count to three by extending their thumb, index finger, and middle finger. If you do that in the US, people there might get horrendously confused, because they do not tend to use their thumbs when signaling the number two. This might lead to frustration, especially in restaurants (trust me). If we could find a way to generalize these two scenarios—maybe by appropriately counting the number of extended fingers—we would have an algorithm that could teach simple hand gesture recognition to not only a machine but also (maybe) to an...

Planning the app


The final app will consist of the following modules and scripts:

  • gestures: A module that consists of an algorithm for recognizing hand gestures. We separate this algorithm from the rest of the application so that it can be used as a standalone module without the need for a GUI.

  • gestures.HandGestureRecognition: A class that implements the entire process flow of hand-gesture recognition. It accepts a single-channel depth image (acquired from the Kinect depth sensor) and returns an annotated RGB color image with an estimated number of extended fingers.

  • gui: A module that provides a wxPython GUI application to access the capture device and display the video feed. This is the same module that we used in the last chapter. In order to have it access the Kinect depth sensor instead of a generic camera, we will have to extend some of the base class functionality.

  • gui.BaseLayout: A generic layout from which more complicated layouts can be built.

  • chapter2: The main script for the chapter...

Left arrow icon Right arrow icon

Description

Design and develop advanced computer vision projects using OpenCV with Python About This Book Program advanced computer vision applications in Python using different features of the OpenCV library Practical end-to-end project covering an important computer vision problem All projects in the book include a step-by-step guide to create computer vision applications Who This Book Is For This book is for intermediate users of OpenCV who aim to master their skills by developing advanced practical applications. Readers are expected to be familiar with OpenCV’s concepts and Python libraries. Basic knowledge of Python programming is expected and assumed. What You Will Learn Generate real-time visual effects using different filters and image manipulation techniques such as dodging and burning Recognize hand gestures in real time and perform hand-shape analysis based on the output of a Microsoft Kinect sensor Learn feature extraction and feature matching for tracking arbitrary objects of interest Reconstruct a 3D real-world scene from 2D camera motion and common camera reprojection techniques Track visually salient objects by searching for and focusing on important regions of an image Detect faces using a cascade classifier and recognize emotional expressions in human faces using multi-layer peceptrons (MLPs) Recognize street signs using a multi-class adaptation of support vector machines (SVMs) Strengthen your OpenCV2 skills and learn how to use new OpenCV3 features In Detail OpenCV is a native cross platform C++ Library for computer vision, machine learning, and image processing. It is increasingly being adopted in Python for development. OpenCV has C++/C, Python, and Java interfaces with support for Windows, Linux, Mac, iOS, and Android. Developers using OpenCV build applications to process visual data; this can include live streaming data from a device like a camera, such as photographs or videos. OpenCV offers extensive libraries with over 500 functions This book demonstrates how to develop a series of intermediate to advanced projects using OpenCV and Python, rather than teaching the core concepts of OpenCV in theoretical lessons. Instead, the working projects developed in this book teach the reader how to apply their theoretical knowledge to topics such as image manipulation, augmented reality, object tracking, 3D scene reconstruction, statistical learning, and object categorization. By the end of this book, readers will be OpenCV experts whose newly gained experience allows them to develop their own advanced computer vision applications. Style and approach This book covers independent hands-on projects that teach important computer vision concepts like image processing and machine learning for OpenCV with multiple examples.

Who is this book for?

This book is for intermediate users of OpenCV who aim to master their skills by developing advanced practical applications. Readers are expected to be familiar with OpenCV&apos

What you will learn

  • Generate real-time visual effects using different filters and image manipulation techniques such as dodging and burning
  • Recognize hand gestures in real time and perform hand-shape analysis based on the output of a Microsoft Kinect sensor
  • Learn feature extraction and feature matching for tracking arbitrary objects of interest
  • Reconstruct a 3D real-world scene from 2D camera motion and common camera reprojection techniques
  • Track visually salient objects by searching for and focusing on important regions of an image
  • Detect faces using a cascade classifier and recognize emotional expressions in human faces using multi-layer peceptrons (MLPs)
  • Recognize street signs using a multi-class adaptation of support vector machines (SVMs)
  • Strengthen your OpenCV2 skills and learn how to use new OpenCV3 features

Product Details

Country selected
Publication date, Length, Edition, Language, ISBN-13
Publication date : Oct 19, 2015
Length: 230 pages
Edition : 1st
Language : English
ISBN-13 : 9781785289866
Category :
Languages :
Tools :

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
OR
Modal Close icon
Payment Processing...
tick Completed

Billing Address

Product Details

Publication date : Oct 19, 2015
Length: 230 pages
Edition : 1st
Language : English
ISBN-13 : 9781785289866
Category :
Languages :
Tools :

Packt Subscriptions

See our plans and pricing
Modal Close icon
$19.99 billed monthly
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Simple pricing, no contract
$199.99 billed annually
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just $5 each
Feature tick icon Exclusive print discounts
$279.99 billed in 18 months
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just $5 each
Feature tick icon Exclusive print discounts

Frequently bought together


Stars icon
Total $ 153.97
Learning OpenCV 3 Computer Vision with Python (Update)
$54.99
OpenCV with Python Blueprints
$43.99
OpenCV with Python By Example
$54.99
Total $ 153.97 Stars icon
Banner background image

Table of Contents

8 Chapters
1. Fun with Filters Chevron down icon Chevron up icon
2. Hand Gesture Recognition Using a Kinect Depth Sensor Chevron down icon Chevron up icon
3. Finding Objects via Feature Matching and Perspective Transforms Chevron down icon Chevron up icon
4. 3D Scene Reconstruction Using Structure from Motion Chevron down icon Chevron up icon
5. Tracking Visually Salient Objects Chevron down icon Chevron up icon
6. Learning to Recognize Traffic Signs Chevron down icon Chevron up icon
7. Learning to Recognize Emotions on Faces Chevron down icon Chevron up icon
Index Chevron down icon Chevron up icon

Customer reviews

Top Reviews
Rating distribution
Full star icon Full star icon Full star icon Full star icon Half star icon 4.4
(10 Ratings)
5 star 70%
4 star 20%
3 star 0%
2 star 0%
1 star 10%
Filter icon Filter
Top Reviews

Filter reviews by




Amazon Customer May 17, 2016
Full star icon Full star icon Full star icon Full star icon Full star icon 5
"OpenCV with Python Blueprints" is a great introduction to intermediate and advanced OpenCV concepts. Technical, but not too much. It's really helpful if you want to build your own functional apps.The book focuses on the practical side of things: Every chapter describes a dedicated, stand-alone project from A-Z. I'm big on 3D scene reconstruction, so I really enjoyed the chapter about optic flow and structure-from-motion. For a book that has to cover a lot of breadth, I was impressed by how much detail was contained in just one chapter. You get a really good value here.I also liked the structure of the book a lot: At the beginning of a chapter, the author tells you the goal of the project and outlines how exactly to get there step-by-step. If there is some theory or background information you need, he will introduce the concepts and explain them using a simple example. Very easy to follow. Every chapter finishes with the finished app, so you know exactly what you'll get in the end.I highly recommend this book to anyone with the requisite background trying to get a better understanding of the more advanced OpenCV tools.
Amazon Verified review Amazon
Nick Nov 17, 2015
Full star icon Full star icon Full star icon Full star icon Full star icon 5
This book covers multiple computer vision algorithms used in the industry today. Beyeler focuses on a practical approach by providing detailed Python code to perform image manipulation, augmented reality, object tracking, 3D scene reconstruction, statistical learning, and object categorization.I particularly liked the last chapter on facial expression recognition using Haar Cascade Classifiers and multi-layer perceptrons, where the authors described very clearly their implementation and usage with openCV. Finally, having access to all the source code on Github is great and really useful in order to quickly test the models, and modify them yourself.I recommend this book for people who already have some experience with computer vision and who want to improve their skills by developing advanced practical applications with OpenCV.
Amazon Verified review Amazon
charliesixpack Jun 13, 2016
Full star icon Full star icon Full star icon Full star icon Full star icon 5
I cannot get the code to work with either of the platforms I have tried. These are common platforms I would expect the code to be compatible with.Macbook Pro running OS X 10.11.5: The code in the book requires the wxPython package. The download "wxPython3.0-osx-3.0.2.0-cocoa-py2.7.dmg" simply does not install. This problem is documented and unresolved. An error message appears during install about the downloaded software not being found by the installer. Without this package the code is useless. This is not a problem with the book but without the package the book is useless.Raspberry Pi 3 with raspberry pi camera: The opencv function VideoCapture(0) does not work with the Raspberry Pi camera. So I spent a couple of days trying to get the equivalent picamera.capture in conjunction with picamera.array.PiRGBArray to work with the code by producing an equivalent numpy three dimensional array for the frame the code would work with. My syntax and data types check out but I get a runtime error "Failed to gain raw access to bitmap data." What does that mean?I am frustrated and disappointed that neither of my platforms can use the code in the book. If the book code worked with tweaking I would give it 5 stars.Update: I have finally gotten a wxPython frame to display from the pi camera on the Raspberry Pi by using the hint from G10DRAS on [...]So I am off and running and have revised my rating.
Amazon Verified review Amazon
Amazon Customer Mar 25, 2016
Full star icon Full star icon Full star icon Full star icon Full star icon 5
Excellent book to build practical OpenCV projects! I'm still relatively new to OpenCV, but all examples are well laid out and easy to follow. The author does a good job explaining the concepts in detail and shows how they apply in real life. As a professional programmer, I especially love that you can just fork the code from GitHub and follow along. Strongly recommend to readers with basic knowledge of computer vision, machines learning, and Python!
Amazon Verified review Amazon
JrB May 11, 2018
Full star icon Full star icon Full star icon Full star icon Full star icon 5
There a lot of information in this book. It is to the point. I think it is great for a beginner to semi-advanced users. Plenty of examples to help you really understand what is going on.
Amazon Verified review Amazon
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

How do I buy and download an eBook? Chevron down icon Chevron up icon

Where there is an eBook version of a title available, you can buy it from the book details for that title. Add either the standalone eBook or the eBook and print book bundle to your shopping cart. Your eBook will show in your cart as a product on its own. After completing checkout and payment in the normal way, you will receive your receipt on the screen containing a link to a personalised PDF download file. This link will remain active for 30 days. You can download backup copies of the file by logging in to your account at any time.

If you already have Adobe reader installed, then clicking on the link will download and open the PDF file directly. If you don't, then save the PDF file on your machine and download the Reader to view it.

Please Note: Packt eBooks are non-returnable and non-refundable.

Packt eBook and Licensing When you buy an eBook from Packt Publishing, completing your purchase means you accept the terms of our licence agreement. Please read the full text of the agreement. In it we have tried to balance the need for the ebook to be usable for you the reader with our needs to protect the rights of us as Publishers and of our authors. In summary, the agreement says:

  • You may make copies of your eBook for your own use onto any machine
  • You may not pass copies of the eBook on to anyone else
How can I make a purchase on your website? Chevron down icon Chevron up icon

If you want to purchase a video course, eBook or Bundle (Print+eBook) please follow below steps:

  1. Register on our website using your email address and the password.
  2. Search for the title by name or ISBN using the search option.
  3. Select the title you want to purchase.
  4. Choose the format you wish to purchase the title in; if you order the Print Book, you get a free eBook copy of the same title. 
  5. Proceed with the checkout process (payment to be made using Credit Card, Debit Cart, or PayPal)
Where can I access support around an eBook? Chevron down icon Chevron up icon
  • If you experience a problem with using or installing Adobe Reader, the contact Adobe directly.
  • To view the errata for the book, see www.packtpub.com/support and view the pages for the title you have.
  • To view your account details or to download a new copy of the book go to www.packtpub.com/account
  • To contact us directly if a problem is not resolved, use www.packtpub.com/contact-us
What eBook formats do Packt support? Chevron down icon Chevron up icon

Our eBooks are currently available in a variety of formats such as PDF and ePubs. In the future, this may well change with trends and development in technology, but please note that our PDFs are not Adobe eBook Reader format, which has greater restrictions on security.

You will need to use Adobe Reader v9 or later in order to read Packt's PDF eBooks.

What are the benefits of eBooks? Chevron down icon Chevron up icon
  • You can get the information you need immediately
  • You can easily take them with you on a laptop
  • You can download them an unlimited number of times
  • You can print them out
  • They are copy-paste enabled
  • They are searchable
  • There is no password protection
  • They are lower price than print
  • They save resources and space
What is an eBook? Chevron down icon Chevron up icon

Packt eBooks are a complete electronic version of the print edition, available in PDF and ePub formats. Every piece of content down to the page numbering is the same. Because we save the costs of printing and shipping the book to you, we are able to offer eBooks at a lower cost than print editions.

When you have purchased an eBook, simply login to your account and click on the link in Your Download Area. We recommend you saving the file to your hard drive before opening it.

For optimal viewing of our eBooks, we recommend you download and install the free Adobe Reader version 9.