As a final project for our Autonomous Robots class, my partner and I
decided to program a NAO humanoid robot to use an elevator. It used
colored beacons and cardboard patches placed inside the elevator for
localization, and asked a human for help with pressing the elevator
buttons. The processes was completely autonomous.
Here is a video of the NAO taking an elevator (simulated lab environment):
We did most of our development and testing in the lab to avoid occupying
a busy elevator. This video shows our demo running in the lab. We used a
cardboard panel instead of a door, and mimicked the size and dimensions
of an actual elevator.
Click on an image to enlarge.
We used an Aldebaran NAO humanoid robot for this project. They are
small but versatile robots, capable of walking and speaking, and
arbitrary joint movements. The NAO has two on-board cameras which we
used as our primary means of sensing the environment.
To use the elevator, the robot used a series of states in which it
performed different actions. First, it asked us to push a button.
Once it saw that we pushed the right button, it looked out for the
elevator door to open. When it did open, it walked inside the
elevator, and went to the button panel where it requested the
correct floor button to be pushed. We pushed the button, and the NAO
went to the middle of the elevator to wait for the doors to open
again. If the doors opened on the desired floor, it walked out of
To localize itself in the elevator, we had the NAO track two beacons
in the back-right and front-left corners, and a blue cardboard patch in
the back of the elevator. Using vision, the robot was able to determine
its angle and distance from these landmark objects, thus being able to
deduce its location in the elevator as needed.
Each floor was identified by a unique beacon. When the doors opened,
if the NAO saw the correct floor beacon, it would exit the elevator.
Detecting color regions in the video frames was done using a color
segment detection algorithm from
We implemented the algorithm in C++ and used it on the NAO's onboard
computer to find blobs of the same color. We used this method for
finding the blue cardboard patch, for detecting beacons, and for
seeing the elevator buttons.
To walk up to target beacons or specified locations, the robot used
PID control. The
PID (Proportional-Integral-Derivative) controller
code goes between the sensor and the actuator, updating the walk
speed and angle to correct for any potential errors in the motors and
sensor readings. It made walking smoother and more accurate.
To avoid having to work in a real, busy elevator, we did most of our
development and testing in the lab. We took measurements of the elevator,
and positioned the visual landmarks in the lab as if they had been
laid out in the real elevator. To the robot, the actual location did not
matter - the visual markers were the same in the lab as in the real world.
When we finished the implementation, we tested the robot in the
real elevator. Out of ten trials, it was able to complete four
completely autonomously. The major point of failure was the poor
lighting in the elevator. This caused the NAO's cameras to
not see some of the colors of various landmarks, thus failing
to detect critical positioning queues.
To improve this project, we would have to focus on using more robust
and reliable vision algorithms for localization and for detecting the
elevator buttons and numbers. Using colors works only in well-lit
situations, and requires the elevator to be outfitted with beacons
and visual landmarks before the robot can use it. With a better
understanding of computer vision, it may be possible to modify our
project so that it would be able to use the elevator without using
any visual markers other than the real elevator itself.