Mastering OpenCV with Practical Computer Vision Projects: This is the definitive advanced tutorial for OpenCV, designed for those with basic C++ skills. The computer vision projects are divided into easily assimilated chapters with an emphasis on practical involvement for an easier learning curve.

In this section we will create a demo application for iPhone/iPad devices that will use the OpenCV (Open Source Computer Vision) library to detect markers in the camera frame and render 3D objects on it. This example will show you how to get access to the raw video data stream from the device camera, perform image processing using the OpenCV library, find a marker in an image, and render an AR overlay.

We will start by first creating a new XCode project by choosing the iOS Single View Application template, as shown in the following screenshot:

Now we have to add OpenCV to our project. This step is necessary because in this application we will use a lot of functions from this library to detect markers and estimate position position.

OpenCV is a library of programming functions for real-time computer vision. It was originally developed by Intel and is now supported by Willow Garage and Itseez. This library is written in C and C++ languages. It also has...

Each iOS application contains at least one instance of the UIViewController interface that handles all view events and manages the application's business logic. This class provides the fundamental view-management model for all iOS apps. A view controller manages a set of views that make up a portion of your app's user interface. As part of the controller layer of your app, a view controller coordinates its efforts with model objects and other controller objects—including other view controllers—so your app presents a single coherent user interface.

The application that we are going to write will have only one view; that's why we choose a Single-View Application template to create one. This view will be used to present the rendered picture. Our ViewController class will contain three major components that each AR application should have (see the next diagram):

The video source is responsible for providing new frames...

A marker is usually designed as a rectangle image holding black and white areas inside it. Due to known limitations, the marker detection procedure is a simple one. First of all we need to find closed contours on the input image and unwarp the image inside it to a rectangle and then check this against our marker model.

In this sample the 5 x 5 marker will be used. Here is what it looks like:

In the sample project that you will find in this book, the marker detection routine is encapsulated in the MarkerDetector class:

/**
 * A top-level class that encapsulate marker detector algorithm
 */
class MarkerDetector
{
public:
  
  /**
   * Initialize a new instance of marker detector object
   * @calibration[in] - Camera calibration necessary for pose estimation.
   */
  MarkerDetector(CameraCalibration calibration);
  
  void processFrame(const BGRAVideoFrame& frame);
  
  const std::vector<Transformation>& getTransformations() const;
  
  protected:
  bool findMarkers...

Augmented Reality tries to fuse the real-world object with virtual content. To place a 3D model in a scene, we need to know its pose with regard to a camera that we use to obtain the video frames. We will use a Euclidian transformation in the Cartesian coordinate system to represent such a pose.

The position of the marker in 3D and its corresponding projection in 2D is restricted by the following equation:

P = A * [R|T] * M;

Where:

After performing the marker detection step we now know the position of the four marker corners in 2D (projections in screen space). In the next section you will learn how to obtain the A matrix and M vector parameters and calculate the [R|T] transformation.

So, by now you already know how to find the markers on the image to calculate their exact position in space, relative to the camera. It's time to draw something. As already mentioned, to render the scene we will use OpenGL functions. 3D visualization is a core part of Augmented Reality. OpenGL provides all the basic features for creating high-quality rendering.

Understanding the principles of the rendering system will give you the necessary experience and knowledge to use these engines in the future or to write your own.

In this chapter we learned how to create a mobile Augmented Reality application for iPhone/iPad devices. You gained knowledge on how to use the OpenCV library within the XCode projects to create stunning state-of-the-art applications. Usage of OpenCV enables your application to perform complex image processing computations on mobile devices with real-time performance.

From this chapter you also learned how to perform the initial image processing (translation in shades of gray and binarization), how to find closed contours in the image and approximate them with polygons, how to find markers in the image and decode them, how to compute the marker position in space, and the visualization of 3D objects in Augmented Reality.