PhDSupervision:Alessandro Stranieri

---

Ph.D. Student:	Alessandro Stranieri
Ph.D. started on:	January 4th, 2010
Phone number (office):	+32-2-650 31 70
Phone number (mobile):	+32 488 98 51 37

---

Week 5

Journal

Monday, March 12^th

Work on the image acquisition code for the arena cameras. Each camera contains a thread, so that acquisition can work separately on each core.

Tuesday, March 13^th

Started to write the code to configure the cameras from XML. I also roughly implemented the C++ code to performed a timed acquisition of the frames, in order to force the acquisition at a specified frame-rate. I tested this implementation and it seems to work, the program is able to make a frame available within the time specified. This means that given a desired frame rate of 4 fps, each new frame is available within a quarter of a second. Despite that, the acquisition time I recored seemed anyway longer than expected. This might due to the fact that the switch hasn't been still properly tuned and that the network interface on the PC is slower than the one which is going to be actually used.

Plans for the next week

Arena tracking system

If we will have received the camera screws, we will install the remaining 4 cameras. At this point I will start performing acquisition speed tests, meaning that I will test whether a system of 4 cameras, each managed by a single thread can meet the specified timing requirements. Once this has been assessed I will test the marker detection code.

Week 4

Journal

Monday, March 5^th

Completed the first draft of the UML for the vision packages in ARGoS. Fixing the scripts for the arena camera calibration.

Tuesday, March 6^th

Installed the first camera in the arena. We tested the connection through the switch, and it seems to work fine. I also performed a calibration test, with which I am having some issues, but they should be easy to fix.

Wednesday, March 7^th

I worked on the first experiment design. Formerly, we wanted to work on an object recognition problem. After some considerations, I decided to conceive the first experiment instead to an image segmentation and shape estimation problem. The main reasons would be the following:

• object recognition is a difficult problem and should be performed once the object in the image has been segmented out from the background;
• segmentation is already an interesting problem. A multiple robot's system can benefit from knowing the position and the shapes of nearby objects. A single robot is not able to localize an unknown object from its image, nor it is able to

construct its representation. Furthermore, objects occluding each other can be potentially perceived as the same one. The objective of the first experiment will be to have a multiple robots perceiving different objects in the environment, successfully discriminate among them and localize them. All the robots must collaborate in order to create a shared representation of the environment. The performance of the experiment can be measured with respect to three aspects:

• Adherence of the constructed representation to the real world;
• Comparison with the result obtained by a single robot suing structure from motion
• Quantity of information exchanged between the robots.

The first step to make, is to design an experimental environment that could suit the purposes of the experiment. For the moment I would think of a simple solution, consisting of a small set of robots ( 4/5 ) placed in an environment populated by a small set of objects. The objects should be placed so that for some of the robots, objects partially or entirely cover one another. Having acquired an image on each of the robots, I could then be able to work offline.

In the next phase I would work offline on the images acquired. The objective would be to test different features detectors to see which ones are more suited to identify the same portion of an object imaged from different view points. This would then enable me to study how to fuse the information exchanged by neighboring robots to perform the object segmentation from the background.

Thursday, March 8^th

Set-up the ceiling camera on the robots in vertical position, and calibrated the internal parameters of each camera.

Friday, March 9^th

Took first set of picture with the footbot. I set-up a very small scenario in which five robots are displaced at different positions in the nearby of three prays and viewing with the ceiling camera connected in frontal position. I recorded positions and orientations of footbots and of the preys. The idea is to test the features detectors implemented in OpenCV to see whether same parts in the real world can be matched across several views of them. The same in the real world, can appear quite different when imaged by different cameras at different views. If this point can be detected and matched in two or more images, the position of this point in the real world can be estimated. Two or more robots, can exchange the description of the points they detected and try to match them. Once the points have been matched, the robots can use their relative position and orientation to determine coordinates of the points they detected. This test is important as from it it depends the possibility for the robots to merge the information they extract from the acquired images.

Plans for the next week

Arena tracking system

• Fix calibration procedure
• Install other three cameras
• Perform image acquisition test.

Thesis project

Week 3

Journal

Tuesday, February 28^th

Implemented three scripts to perform the camera internal parameters calibration. The three scripts are used to:

• Create a PostScript file representing the calibration plate to use for calibration. A parameter of this script is the desired size of the plate.
• Interactively acquire a sequence of frames containing from the camera. When in execution, the window displays whether the calibration plate has been detected. In this way the user can acquire only those frames which are useful.
• Extract the camera internal parameters. The scripts opens the image files present in a given directory and runs the calibration plate detection. At the end, it writes the calibration parameters in a file.

Wednesday, February 29^th

Collected 4 papers:

• A Multi-View Probabilistic Model for 3D Object Classes
• A Fast and Robust Descriptor for Multiple-view Object Recognition
• Unsupervised Feature Selection via Distributed Coding for Multi-view Object Recognition
• Towards an efficient distributed object recognition system in wireless smart camera networks

Thursday, March 1^st

Collected 4 papers:

• Coding of Image Feature Descriptors for Distributed Rate-efficient Visual Correspondences.
• Evaluation of Interest Point Detectors and Feature Descriptors for Visual Tracking
• Cooperative Multi-Robot Localization under Communication Constraints
• Distributed Robuts Data Fusion Based on Dynamic Voting

Had a meeting with Mauro. After the meeting, we agreed that the object of the first experiment will be the study of an object detection problem in a Swarm Robotics system. The main vision is to have a group of robots surrounding an object. The swarm of robots is capable of recognizing the object only by fusing the information processed. The design and realization of this experiment brings a series of problems, of which the first one would be in my opinion how to model the uncertainty of the model. This will be addressed in the following two days.

Friday, March 2^nd

Today I had to begin to work on the re-factoring of parts in ARGoS concerning vision. This was requested by a student of IDSIA, and I decided to begin the work, as I will anyway need more flexibility when implementing the image processing operators for the robots.

Plans for the next week

Arena tracking system

The plan is to finish the calibration process for all the cameras that we have in the lab. I will buy the wooden board to use as a support the calibration pattern sheet, run the calibration script on the four cameras and then install them in a 2x2 matrix at the far left side of the arena. Once the installation is done, I will calibrate also the extrinsic parameters. The test for this phase would be the detection of a robot marker in the exact position in the arena coordinate system.

Week 2

Journal

Monday, February 13^th

Studied the Halcon code and examples to perform a calibration. Discussed some preliminary aspects with Dhananjay, who will help me during the installation and calibration process.

Tuesday, February 14^th

Decided the strategy to perform calibration and installation. The process will mainly consist of two phases. The first phase is dedicated to the calibration of the internal camera parameters. This can be done before the cameras are actually installed by acquiring different views of a specific calibration pattern. The second phase is dedicated to the installation of the cameras installation on the ceiling and the computation of the extrinsic parameters. This means calculating the transformation of the position of an object in an image, to the position in the real world, given the known distance of the object from the camera center. This implies that the positioning of the calibration pattern center to a given position in the arena coordinate system be done in a fairly precise way. I agreed with Dhananjay that I will build a vertical pole to which we will attach the frame. This vertical pole will be then positioned at known points in the ground. This means that, we should agree on what is the x, y and origin of the area coordinate system.

Studied a solution for a supervision page on the IRIDIA wiki. A possible way to do it could be the following. I can place a link in the main page of the wiki, to a page called something like “PhD Supervision”. The link is only accessible to the people belonging to a group called “PhDSupervision” to which at the beginning only Marco and Mauro belong to. From the page “PhD Supervision”, one can create a page for each student, readable and editable again only by the people belonging to the group. For the moment I have two issues: The link in the main page, although not accessible, will still be visible to everyone logged in. Maybe I can find a solution for that. In mediawiki, it is easy to restrict access to a page on a group basis, but it seems more complicated on a user basis. There are two ways to proceed quickly, while I see whether something can be done: Including the PhD students in the “PhDSupervision” group. This means every PhD student can view the supervision page of the other students; Creating a group for each student and granting access also to this group. I actually find this solution messy.

Plans for the next week

During the next week I won’t be able to dedicate much time to anything, as I am in Parma for the MIBISOC seminars.

Arena tracking system

Since it doesn’t take a lot of effort, I will prepare all the coded needed for the calibration. Once, I am back, it will only be a matter of creating the calibration pattern and running the software.

Thesis project

Week 1

Journal

Friday, February 3^rd

Bought first batch of material to build the camera supports. Started planning of supports´ shape and structure. Planned camera positioning and marked the position for the cameras along the row closer to the arena windows.

Monday, February 6^th

Bought other material for the camera support. Prepared a master support. Based on this master, manufactured 6 different frames to speed the preparation of the final camera supports.

Tuesday, February 7^th

Had the material bought on Tuesday replaced with a set of properly cut pieces. Finished preliminary work on the support. 16 supports are ready.

Wednesday, February 8^th

Collected and studied 5 papers concerning my project:

• Vision-Based Global Localization and Mapping for Mobile Robots
• Vision-Based Localization and Data Fusion in a System of Cooperating Mobile Robots
• Collective Perception in a Robot Swarm
• Distributed Sensor Fusion for Object Position Estimation by Multi-Robot Systems
• Distributed Multirobot Localzation

The comments on the above-mentioned papers, as well as the next ones, will be given in a separate PDF file, which I will soon make available.

Thursday, February 9^th

Began the creation of a frame grabber for the E-Puck camera. This is needed in order to enable Arne’s and Manuele’s students to work with the front camera on the E-Puck.

Friday, February 10^th

Meeting with Mauro. The main purpose of this meeting was to outline the first steps to take towards the definition of the thesis subject and of the first experiments. We agreed on a set of aspects that have to be addressed and mainly concerning the general message of my work and the settings of the first experiments. As a result of the meeting, we established that the next following two/three weeks will be dedicated to: Elaboration of a clear and sound message that will be the main purpose of my thesis. This means considering the definition of what for the moment I will call Swarm perception and the listing of the related works. Acquisition of a first data-set of images to work on a validation experiment. To put it briefly, the experiment involves a swarm of robots which use vision and local communication to improve the estimation of an object distance. The purpose of this first test is twofold: Study the kind of information that the robots should be able to extract from the acquired images and how much of this information should they exchange with each other; Prove that the idea of improving the capabilities of environment perception can be enhanced by means of a ¨Swarm Robotics¨ approach is worth investigating.

This phase will actually have a further purpose. The vision support in ARGoS needs a work of refactoring aimed at: Ease the introduction of new computer algorithms on the robots; Update the currently used version of OpenCV; Introduce image acquisition capabilities for the E-Puck; [Optional] Allow the uses of computer vision algorithm also in simulation (to be discussed with Carlo).

During this first phase I will propose a possible way of action to address this point. The output will eventually be the steps to be taken in order to accomplish that. If everything goes according to plan, the mentioned steps will be taken in what I and Mauro agreed to be the second phase, which will be dedicated to: Refactoring of the vision support in ARGoS; Coding of the new software modules that will be used in my work.

Given the bigger priority of the first phase, I will not comment any further on the second one.

Plans for the next week

As for this past week, I will dedicate Monday and Tuesday to the work on the tracking system, while the rest of the week will be dedicated to the work on my project.

Arena tracking system

With the help of Ali, I will begin the installation of four cameras in a 2x2 configuration. Before this, we will mark once and for all the positions of all cameras on the ceiling wooden structure. At the same time, I will start working on: A vision-based procedure to aid the camera positioning. This means that I will try to use the camera itself and software to aid the alignment of the camera with the ground plane. The strategy for the camera calibration. This step will surely involve the construction of a proper calibration structure (a board with a pre-defined pattern)

This second point is likely to take also the following week.

Thesis project

Regarding my project, I plan to dedicate the rest of the week to the following activities: Creation of a document where I define the focus of my studies. In the beginning this document will mainly contain: The definition of my study and the comments on the long term vision My comments on the related works I have been reading and collecting so far Creation of a second document, where I will describe my first experimental work.

I would consider the first document as an incubation draft of what could go in the end in the introduction and state of the art of my thesis. The second document instead would eventually grow into a conference paper. These two documents will be prepared in LaTeX, but I plan to share the PDF version in Google docs.