Real-time speed estimation of cars with OpenCV

Posted by Armin on Friday, April 15, 2016

Little car icon

Background and goal

I live in a residential neighborhood where the speed limit is 25 miles/hour (ca. 40 km/h). I find this to be high for a residential neighborhood, but 25 mph is the norm here in the USA. Most of Europe, in comparison, has a history of posting 30 km/h in similar situations. (According to Wikipedia: "On September 1, 1992, the city of Graz, Austria, became the first European city to implement a city-wide 30 km/h limit on all roads.")

In any case, I feel that many cars are going even faster than the posted 25 mph, especially during rush hours. To substantiate my gut-feeling, I decided to measure vehicle speeds on the street in front of our house. I couldn't really justify spending a lot of money on a radar system but wanted to have a cheap way of getting some statistics, with tools and cameras I already have at hand. This is a work in progress, and while the vehicle speed estimation is already working, I have not had a chance to set up a dedicated camera and computer system to actually gather statistics.

Video screenshot of the first test version. Top left: original video frame. Top right: Perspective corrected according to calibration rectangle of known dimensions. Detected vehicle tracks are shown as an overlay.Bottom left: Instantaneous and cumulative average speed of currently detected vehicles. Bottom right:Average speeds over the calibration area for the last few vehicles.

Ideally, I would like to use a camera with a slightly wider lens to give the Kalman filter more time for conditioning.

Implementation

Speed estimation is done in a C++ based program, using the OpenCV library for background detection and various transformations. I'm using the Cinder library for visualization, because I wanted to learn it, it's beautiful, and it has some really neat features such as "cinder::params::InterfaceGl", an easy-to-use wrapper for the AntTweakBar GUI library.

Program logic

Image conditioning

Figure 1:

A video frame is captures either from a file or from a camera ( A-1). A rectangle of known dimensions drawn on the street (marked during a calibration procedure with white dots as seen in B allows for a perspective transform (A-2) of the acquired image. This transform helps in getting constant pixel/second speeds for a vehicle travelling with constant speed over the calibration area. The perspective corrected frame is seen in the top right of C

An OpenCV background segmentation algorithm (Mixture of Gaussians Segmenter, BackgroundSubtractorMOG2) is then run on the transformed image (A-3). Anything moving will appear as a white blob in the thresholded image. These white blobs are then sent to a contour detection routine (A-4, which returns the center coordinates of each blob. These center coordinates are then sent to the Vehicle Tracker class (A-5), which determines whether they belong to a vehicle, and if so, what the instantaneous and average vehicle speed is.

VehicleTracker class

Center coordinates for moving contours detected in the previous step are fed to the "VehicleTracker" object. As the name implies, the VehicleTracker keeps track of all the vehicles that might appear at any given time. To do so, it uses a Kalman tracker for each detected vehicle track, and performs some rudimentary estimation of whether the object present is a car or not.

Figure 2: A simple overview of what the VehicleTracker class does during program execution.

Code

I currently don't have time to continue on this project. I've opened the bitbucket repository of my XCode project, and you can find the relevant code in the "Source" page.

Sources
Tags: 
C++
OpenCV
computer vision
Cinder
code
Category: 
Code

Comments

Tim replied on

Hi Armin

Absolutely love what you've achieved with this, and I'd really like to know more. I've got a very similar problem where the purchase of a radar module is getting a bit silly money-wise, and I'm wondering about using OpenCV for estimating traffic speed in our village in the UK. I'm new to OpenCV and so I'm working hard through the docs and tutorials to try to get a grip of how to apply the transforms and filters on a video stream. Anything (however presentable) you could share would be a great help.

Armin replied on

Hi Tim, I haven't worked on this project since I published this post, so unfortunately it is still not very advanced. I've only tried it for the street right outside my house, so I don't know how robust it is when faced with different situations. It's also using the libcinder library for the GUI, which is a bit more complicated than necessary during the testing phase. (I wish I had stuck to the OpenCV highgui, but it's easy enough to reuse the classes to do just that for faster prototyping.) Anyways, I can share the project with you on bitbucket so you can see the code. Just sent me an email address (or bitbucket user name). You can reach me through the contact form. Cheers, Armin

Tim replied on

Thanks for the quick reply and comments Armin... It looks like the opencv gui may be the simpler way to go. Thanks for the upload... My username is thechapel.

I'll let you know how I get on.

Armin replied on

I've shared the repository with you. Sorry it's such a mess. It's an XCode project, and the relevant files are in the "xcode" directory if you go to the "source" view. Most of the classes that do stuff have their own file. For example, "Transformation.hpp" and "Transformation.cpp" describe the object that does the image perspective transformation; the "TrackerKalman" files describe the Kalman filter, etc. My next goal is to change the Kalman filter so it takes the size of the detected "blob" into account. If the blob size suddenly increases a lot (e.g. when two cars are passing each other and are seen as one object), this measurement shouldn't be taken as seriously. I'm going to rewrite it all to use the OpenCV highgui for showing the images, because the whole thing would be much simpler then. (You'll see a header called "CinderOpenCVBridge.hpp" which performs some convenience functions to translate between OpenCV and Cinder. It's basically just copied from the OpenCV "cinder block".) Don't get me wrong, I LOVE libcinder, but here it's a bit overkill...

Armin replied on

I just saw that the code that actually defines the Cinder app is located in the "src" folder, named "VehicleTracker_v03App.cpp". (The way my files are organized in XCode's workspace do not actually reflect where they are in the file structure.) This would be the code that sets up all the other classes to perform the tracking and show the results...

Shubham Singh replied on

Hi Armin,
im shubham,a third year engineering student from india.im doing a similar project and needed help to get started.can you please share the code with me.my email id is- shubham88fru@yahoo.com
thanks in advance :)

Armin replied on

Hi Shubham. I sent you an invitation through my bitbucket account. Hope it helps. - Armin

Agi replied on

Hi Armin,
I was scanning the internet for a similar application, for the very same problem in my neighborhood
I live on busy street, and the vehicles constantly exceed the speed limit, I would like to use the tool to track the vehicle speed and make the community aware of the growing problem. Your code can give me a head start, will you be able to share it

Armin replied on

Sorry I didn't get back to you earlier! Even though I never got a chance to clean up the code as I had hoped, I opened the bitbucket repository so you can take a look. Hope it helps.

Azfar Alsukor replied on

Hi Armin,
I am working on a similar project with same purpose too. However, I am new to OpenCV and above all, Kalman Filter really is a pain. Can you share me your code and any advise on learning about Kalman Filters and how to 'tune' it. My email is azfaralsukor@gmail.com Thanks you very much!

Armin replied on

Hi! Sorry I didn't have time to respond until now. I'm overwhelmed with other things, so this pet project hasn't progressed in months. I've opened the bitbucket repository just now, and will post a link to it in the post above. Can't help with tuning, as I'm still trying to figure that one out myself. (I really want to add another dimension to the Kalman filter that takes into account the area of the blob detected.)

Armin replied on

Hi Eddy! My original plan was to create a cleaner version of what I have so far (using OpenCV's highgui instead of libcinder), but I currently don't have time to work on the project. I just opened up the Bitbucket repository, so anyone can take a look at the (messy) code :)

Esraa Gamal replied on

Hi Armin ,
i’m working on the same topic so i’m badly need your help for developing vehicle speed detector.. can you pleeeese please send me the source code to esraagamal117@gmail.com
thanks in adnance

Armin replied on

Hi, I've put all the code on bitbucket (https://bitbucket.org/amphioxus/vehicletracker3). It's just a copy of my XCode project, so it's a bit of a mess. But just look in the "Source" tab, and then the "XCode" folder. You'll find the code for all the relevant classes (BackgroundSubtractor, TrackerKalman, AssignmentProblemSolver, etc.) in there. And the "VehicleTracker_v03App.cpp" in the "src" folder shows you how all these classes are set up and used. Good luck!

Esraa Gamal replied on

hello Armin ,
i hope you are fine :) . i have a problem with linking (error LNK1104: cannot open file 'libboost_filesystem-vc120-mt-gd-1_58.lib' ) so i search about the error and i see that i should download C++ Boost libraries and include them . i use visual studio 2013 on windows and i download so many versions of that library but with no difference . i followed all links and tutorials but it's generating more LNk errors .
can you give me a link or tutorial that you are followed in your project to solve boost library linking error . i'd be so happy , if you take my question in consideration (this problem took me a 5 days till now) . thanks in advance
my mail is esraagamal117@gmail.com

Armin replied on

I think boost is only required for the libcinder library, which is the part that draws the images onto the screen. Maybe you could keep it simpler, and just use OpenCV's highgui for displaying stuff. You'd start by creating a basic highgui-based application, and add all the image-detector, kalman-filter, etc. one step at a time. I don't have time any more to work on this project, so unfortunately I don't have time to try any of this out myself. (I had hoped to rewrite it using simpler display methods, but you know how it goes...)

Alex replied on

Hi Armin,
im Alex and doing a similar project and needed help to get started.can you please share the code with me.my email
[...redacted...]
thanks in advance