# The COTBLEDTCID approach to object detection and pose estimation, Part IV – Transformation

## Introduction

With the last step we know where the bottom edges of the pawn are located on the image, we just need to find a way to transform the coordinates of those pixels into game field coordinates.

Given a point $p'$ from the Image plane, we’d like to transform it into $p$ from the Game field plane. We can write:

$p = H \cdot p'$

We observe that straight lines are kept straight, thus H is called the homography matrix which can be computed if at least 4 different matching points are given for both planes. $(p1 \leftrightarrow p1', p2 \leftrightarrow p2', p3 \leftrightarrow p3', p4 \leftrightarrow p4')$

It’s worth noticing that both $p$ and $p'$ points are given in homogeneus coordinates.

# The COTBLEDTCID approach to object detection and pose estimation, Part III – Blob Edge Detection

## Introduction

There is still too much information we do not need on the B&W image we got on the last step. That’s why we need to extract the features we do need. One way of accomplishing this is by performing a connected component analysis in binary images, aka blob labelling. However as you’ll will see, this method is not completely adapted to our needs, so a new approach is proposed: Blob Edge Detection.

# The COTBLEDTCID approach to object detection and pose estimation, Part II – Colour Thresholding

## Introduction

Images are usually too complex to be treated by computers as is. In most cases, they have to be enhanced and simplified before any algorithm can be applied on them. Fortunately for us, the rules of the contest specify that yellow is the colour of pawns and figures we are looking to detect.

Generally speaking, the purpose of Colour Segmentation is to extract information from an image by grouping similar colours. In our algorithm, we implemented colour segmentation by thresholding yellow colours, that is to say, the computer builds a black and white image from the original image where white colour represents yellow colour and black is everything else. This is called colour thresholding.

The problem is, as you may have already guessed, the notion of “yellow” colour. For humans, it is relatively easy to tell whether a colour belongs to a group of colours, but for computers this is a whole different story.

# The COTBLEDTCID approach to object detection and pose estimation, Part I – Preface

## Introduction

A fancy acronym that stands for the process of COlour Thresholding, Blob Edge Detection, Transformation and CIrcle Detection used for locating 3D objects on a plane. The next series of posts will explain the software algorithms used by ClubElek on 2011 to achieve computer vision. The problems we faced, the solutions we implemented and most importantly what we have learned by doing this project.

These posts are targeted to a wide audience with some background in maths and preferably some background in computer vision. Mostly because there are some maths and magic behind the algorithms used, but I’ll try to keep them as simple and clear as possible. Should you have questions or remarks do not hesitate to comment !

This software was designed to detect “pawns” and “figures” defined by the Eurobot 2011 rules and was demoed during “Industrie Lyon” from 5 april to 8 april 2011. Before continuing reading this post you should read the summary about the rules of the contest so you don’t get lost.

Besides the 5 previously mentioned steps, 2 other steps were necessary before attempting any computer recognition: terrain calibration and colour calibration. These 2 processes will be explained on separate posts as they are far more complex than the COTBLEDTCID itself.

After the pawns’ positions had been detected by the means of COTBLEDTCID, they were sent wirelessly to the robot through a XBee connection.

• The software should be used in a real-time environment. The fastest the algorithm, the better.
• The software should be easy to use and fast to configure. (Teams have only 1 minute and 30 seconds before a match to completely set up the robot and its peripherals).

## Hardware set-up

• A Fit-PC-2 disk-less and fan-less computer running a customized ubuntu version controlled through ssh.
• 3 identical Microsoft LifeCam Cinema webcams. Why 3 cameras should you ask. Well, during a match, there are two robots that constantly move around the table and obfuscate large parts of the terrain, with 3 cameras chances are we see most of the objects on the playing table at any time.

• Light intensity remains constant during the match and after calibration.
• The cameras do not move during the match.

Both assumptions resulted to be inaccurate but did not affect the result as the detection and pose estimation algorithm is fairly robust.

## Shiny pics

What the computer sees (note that the robot’s game field wasn’t entirely finished by the time):

What the computer understands: (compare the pawns’ position in both images. You may use the red top corner or the black area at the bottom of the image as a reference)

It’s fairly accurate isn’t it ?

## What’s next

In the next section I will explain how the Colour Thresholding works and why we need it (COT for short).

# Capture Video with OpenCV and VideoInput (Windows only)

As you might have already noticed, the internal’s opencv camera interface is far from complete. You can capture video from your camera without a hassle but you’re very limited to what you can do.

For instance, let’s say you have a webcam that can run at HD resolution ( 1280 x 720 px ), you can use the opencv’s class cv::VideoCapture to get the frames but you’re going to have a hard time on getting the full resolution out of your device.

To sort out this problem, one could use specialized libraries. The disadvantage :  they’re not usually multi platform so you’ll find yourself writing classes for every platform where you’re camera device operates.

Theo  developed a very useful video capture library for windows called videoInput.

# 2010 in review

The stats helper monkeys at WordPress.com mulled over how this blog did in 2010, and here’s a high level summary of its overall blog health:

The Blog-Health-o-Meter™ reads Fresher than ever.

## Crunchy numbers

A Boeing 747-400 passenger jet can hold 416 passengers. This blog was viewed about 9,000 times in 2010. That’s about 22 full 747s.

In 2010, there were 12 new posts, growing the total archive of this blog to 33 posts. There were 17 pictures uploaded, taking up a total of 21mb. That’s about a picture per month.

The busiest day of the year was November 29th with 69 views. The most popular post that day was Start coding with OpenCV and VS2010 in no time.

## Where did they come from?

The top referring sites in 2010 were en.wordpress.com, sfml-dev.org, planete-sciences.org, facebook.com, and reddit.com.

Some visitors came searching, mostly for sfml tutorial, sfml game tutorial, sfml tutorials, sfml games, and sfml game.

## Attractions in 2010

These are the posts and pages that got the most views in 2010.

1

Start coding with OpenCV and VS2010 in no time October 2010
5 comments and 1 Like on WordPress.com,

2

Your first SFML game, Part IV: The main loop August 2009

3

Your first SFML game, Part I : Introduction August 2009

4

Your first SFML game, Part II : The board August 2009

5
By Rodrigo Posted in Life

# Brand new videos !

It’s a pleasure to announce the immediate availability of 2 new videos.

The first one was originally made to recruit new members for the ClubElek’s team. It was later modified to be part of a CD we offer to our sponsors. I really think the video made it’s purpose as there’s a troop of new people coming each week to our chamber of secrets ( also known as the ClubElek’s cave) 😉

The second video was made just for fun :P. It’s an assemble of a recorded match made of three different angles.

Now go get some popcorn, take your favourite seat and enjoy ! (PS: Sorry if you don’t completely understand the first video because the text is in French).

# Start coding with OpenCV and VS2010 in no time

It’s usually very time-consuming to start a new project because you have to manually add the libs and include directories. And when there’s a lot of them. It’s a headache.

Fortunately CMake has made this a really easy step. You only create the CMakeLists.txt file where your project properties reside and CMake does the rest.

In this tutorial I’ll try to explain how to get up and running in no time with OpenCV and Visual Studio 2010.

# Eurobot 2011

The ClubElek team will be a proudly 2011 contestant. The rules have just been disclosed and we’re already working as hard as we can to improve the technology we’ve developed and to innovate in new areas.

We’re ready to rock and roll !!

# A new year for ClubElek

A new year starts for Clubelek, that’s why we’ve created a poster to recruit the most ambitious, the most creative and the most motivated people out there. Hope you like it :

Click the image to see it in full resolution.

If you’re interested and you live nearby, don’t hesitate. Come and see what Clubelek is all about !

I cant’ finish this post without thanking Matthieu for his ideas and his advice.

Sources : The very cool robot toy was made by RayKit Designer