Augmented Reality Games

Our Games

Our games platform consists of two old-school revamped games, space invaders and air pong, that you can control with a colored object of your choice via your laptop’s camera. Just choose a colored object, keep it in the red square to initialize your controller color, and start playing one of the two games. To control your spaceship or air pong paddle, move your controller with your hand. The program will detect its position and visualize your motion accordingly.

Have fun playing our game!

Progress and results

We started our game with the intention to create an air pong game that the player can control via OpenCV, a real-time computer vision library. The first version of the game was split up into two parts: the game and the computer vision controller.

First Steps - Air Pong Game

This meant that we had two files that would communicate with each other. However, the communication process itself required some design decisions. Since Python is an object-oriented programming language, we choose the object-oriented design pattern called Model-View Controller.

After a few days of coding, we were able to detect the position of the controller the player was holding and change the cursor’s position accordingly. The controller’s color had to be green in order to be detected by the camera. We further implemented a mirrored live video from the computer’s camera as the background of the game.

In the beginning, the controller detection just worked for one controller and was relatively slow. Air pong game, however, requires two players. Consequently, we had to come up with a better communication between the model and the controller that would allow us to send coordinates of two controllers. Since the live feed for the background was the same picture that was analyzed to detect the controller, our cursor recognition software had to analyze a comparably large picture. We decided to analyze a much smaller picture for the color detection and feed a bigger camera video to the background to better the performance.

To give the player more room to customize the game, we implemented a start page and a small menu. When starting the game, one of the players had to hover over a square to reach the settings. We implemented this step to ensure that the controller was working correctly. In the menu itself, one could choose between five different speeds for the air pong ball by hovering over the desired difficulty.
The collision detection between the ball, the paddles, and the barriers in the game itself turned out to be more difficult than expected. It tooks us a while to figure out that all the objects that needed to collide, needed to inheret from the "pygame.sprite.Sprite"-class. Next we did a lot of investigation to find out that all the objects that need to collide need to have a "rect"-attribute (and that you cannot forget to update the location of this "rect").
As the last step, we added a score counter to the game, made it show the winner.We also give the player the option to replay the game.

More Games! Space Invaders

The whole process of programming became more complicated when we started implementing the second game - space invaders. We were forced to revise our entire architecture to make it more flexible, efficient and have a nice, clear code. This included creating several files for an organized Model-View-Controller structure, implementing state machines to keep track of where the player is within the game, reorganize our classes with descriptive docstrings to make debugging easier.

For the space invaders game itself, we used the same color recognition file to track the player’s controller. The player controls the position of the spaceship by moving their controller horizontally and shoots bullets by holding your controller above the dark-red line.

One of our problems at that point was that the player had to use a green controller in order to play the game. Consequently, we started implementing an initialization screen that allows the player to choose an arbitrary object as a controller. It calibrates and saves the color of the object that the player seeks to use as a controller. The player could finally use any colored object to play the game, as long as the color he uses is not apparent his background.

Architectural Overview

On a grander scale, our game went through two phases: programming the air pong game and adding the space invaders game. In the first phase, we learned a lot about the structure of a single game. In the second phase, we restructured our entire architecture to make iStateMachinePong.pngt more adjustable and easier to enhance. We will try to list a few interesting software design decisions and characteristics that we came up with for our game platform.

State Machine

The air pong game became complex quite quicklScreenshot state machine.pngy. Due to the number of classes and layers that our game had we decided to find a tool that would keep track of the game’s status. Therefore, we decided to implement a state machine. The state machine would simplify the communication between the files and classes.

How does it work? The state machine is a class object in the main file and contains a state attribute in the form of a string. The states are named such as: “homeScreen”, “pong”, “calibration”, etc. and are defined by IF-statements. Once in IF-statement, it initializes its own model, view and controller, defined in a seperate file (each state has its own file). It then keeps running in a while loop, each time updating all the components in the seperate file.

Every game has its own state machine (hence the 'Organizer' class) to keep track where the player is in the game. With the update() function, it updates the model (where all the things are happening), the view (what needs to be showed on the screen) and the controller (which calls the file that tracks your object), based upon the state it is in. In the seperate file, all the classes and functions for that game are defined.

Interesting implementations

Area Surveillance

We made a class CursorRecognition in the cursor.py file with which we can easily create buttons and use areaSurveillance on them. When creating the class, you give it a text, color and time it takes to trigger, after which it can be used to change variables in the same function. We can for example easily change states in the statemachine after the button has been triggered.

Color Recognition

We found some sources that we used for the color recognition and tweaked them to make them our own. The original file searched for all pixels that were green in a certain color range (HSV = Hue, Saturation, Value), found and object and deleted noise. We then tracked the center of the object to use it for our cursor. This is the basic program that gets called several times a second and returns the coordinates for the cursor. We added more functionality by making another program, SetColor.py, that scans the pixels in a rectangle for a certain time and creates a range around it. These values gets passed to the main color recognition program, so we can track every color we want. Of course you'll have the best results by objects that are not reflecting and not part of your surroundings. After the color has been set, a calibration program runs, that will ask for a new color setting if the player fails to hover over all the buttons in time.

Leon, Richard, and Viktor

We programmed this game as part of the final project for a Software Design class at Olin College of Engineering in the fall of 2018.

Richard Ballaux

Hi, my name is Richard. I am an exchange student from Belgium. I am in my junior year in Electrical and Software Engineering at Group T in Leuven. I had a decent amount of knowledge in object oriented Java when I started this class, but I wanted to learn more about this very powerful other language called Python. The thing I enjoyed the most about the class as I look from it now that it is almost over, is that we did so many small projects. We did small toolboxes to learn about usefull random skills like file opening, we did mini-projects to learn how to make interactive programs and to get used to the basics of object oriented programming. And last but not least, the final project, I think we made a really cool project and it was also fun to work on it with an awesome team.

• LinkedIn





Viktor Deturck



I am an exchange student from Belgium, majoring in Electromechanical Engineering. I am now in my senior year (class of 2019) and wanted to do something else, aside from my usual courses. That's why I decided to take Software Design and get to know the wonderful world of programming. I didn't have any noteworthy coding experience on beforehand so I didn't really know what to expect. It was however very interesting and educational and I got used to it pretty quickly. Python is a very easy and forgivable language and with the help of the well thought out structure of the course I became very comfortable with it towards the final project.

Leon Santen



I am an international student and member of the class 2021 at Olin College. I am majoring in robotics and doing sustainable design research. I am curious about most engineering fields, learning about their potential and using them as powerfully integrated tools. I think it is crucial that engineers consider the impact they have on the society and environment. Besides my technical interests, I love taking sociology classes at Wellesley College, playing music, DJing, and any social activities.

• LinkedIn

Pics and Vids

Our games platform consists of two games, space invaders and air pong, that you can control with a colored object of your choice via your laptop’s camera. Just choose a colored object, keep it in the red square to initialize your controller color, and start playing one of the two games. To control your spaceship or air pong padel, move your controller with your hand. The program will detect its position and visualize your motion accordingly.

Have fun playing our game!

Contact

• Twitter
• Facebook
• Instagram
• GitHub

Elements

i = 0;

while (!deck.isInOrder()) {
    print 'Iteration ' + i;
    deck.shuffle();
    i++;
}

print 'It took ' + i + ' iterations to sort the deck.';