Video-based transportation data collection : Différence entre versions

Version du 13 décembre 2012 à 18:21

Methods for the detection, tracking and classification of road users

• Feature-based tracking is the main and a relatively easy detection and tracking algorithm
  • my paper http://n.saunier.free.fr/saunier/stock/saunier06feature-based.pdf and the paper it is based upon http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.49.599&rep=rep1&type=pdf
  • Some slightly older information http://wiki.polymtl.ca/transport/index.php/FeatureBasedTracking
  • Other features: Predator (video, code source)
  • Feature performance comparison http://vision.middlebury.edu/flow/ (with links to implementations)
• Background subtraction: the most common method is based on a mixture of Gaussians and available in OpenCV http://www.ai.mit.edu/projects/vsam/Publications/stauffer_cvpr98_track.pdf; lit review in http://staff.it.uts.edu.au/~massimo/BackgroundSubtractionReview-Piccardi.pdf
  • Emvisi2: A background subtraction algorithm, robust to sudden light changes http://cvlab.epfl.ch/software/emvisi2/index.php
  • kernel density estimation based background subtraction http://cvlab.epfl.ch/software/emvisi2/index.php
• Object detection/classification: http://lear.inrialpes.fr/people/triggs/pubs/Dalal-cvpr05.pdf, available in OpenCV, which can be used for tracking by detection, e.g. in http://www.vision.ee.ethz.ch/showroom/tracking/
  • tracking by detection: http://www.vision.ee.ethz.ch/showroom/tracking/
• Volume estimation: the basic method relies on background subtraction and fitting volumes for the various road users (rectangular cuboid/prism for vehicles and cylinder for pedestrians). Some initial papers are http://www.sciencedirect.com/science/article/pii/S1077314207000392, http://www.springerlink.com/content/tr6558j731331m78/, http://iris.usc.edu/outlines/papers/2005/song-nevatia-iccv.pdf
• Video annotation and semi-automated tracking for performance evaluation

Other resources

• TRB SHRP2 report on Site-Based Video System Design and Development http://onlinepubs.trb.org/onlinepubs/shrp2/SHRP2_S2-S09-RW-1.pdf
• Zu Kim at California PATH http://gateway.path.berkeley.edu/~zuwhan/
• EPFL, in particular Pascal Fua, http://cvlab.epfl.ch/software/index.php
  • Multiple Instance Learning http://vision.ucsd.edu/~bbabenko/project_miltrack.shtml
  • POM: Occupancy map estimation for people detection http://cvlab.epfl.ch/software/pom/index.php
• ETH work http://www.vision.ee.ethz.ch/
  • Online boosting trackers http://www.vision.ee.ethz.ch/boostingTrackers/
  • Work by Leibe and Van Gool http://www.vision.ee.ethz.ch/members/get_member.cgi?id=1 http://www.vision.ee.ethz.ch/publications/pub_readall.cgi?lang=en&year1=&year2=&authors=leibe&keywords=
  • Linear Trajectory Avoidance - A Pedestrian Motion Model http://people.ee.ethz.ch/~pestefan/lta/
• Mohan Trivedi (Computer Vision and Robotics Research Laboratory http://cvrr.ucsd.edu/) and his former student Brendan Morris http://www.ee.unlv.edu/~b1morris/
• Minnesota lab for ITS and video analysis http://airvl.cs.umn.edu/ (work by Masoud, Papanikonikolopoulos et al.)
• Gerard Medioni http://iris.usc.edu/people/medioni/index.html http://iris.usc.edu/USC-Computer-Vision.html
• Greg Mori http://www.cs.sfu.ca/research/groups/VML/MMTrack.html (see references on online tracking
• Ram Nevatia http://iris.usc.edu/Projects/detect/detection.html http://iris.usc.edu/Outlines/Paper-track.html
• Rabaud and Belongie, Counting Crowded Moving Objects http://vision.ucsd.edu/~vrabaud/
• Literature reviews: A Review of Computer Vision Techniques for the Analysis of Urban Traffic http://dx.doi.org/10.1109/TITS.2011.2119372, Object Tracking: A Survey, http://dx.doi.org/10.1145/1177352.1177355
• Computer Vision Algorithm Implementations http://www.cvpapers.com/rr.html (see in particular object detection and tracking)
• Minnesota: Practical Methods for Analyzing Pedestrian and Bicycle Use of a Transportation Facility www.lrrb.org/pdf/201006.pdf
• My other delicious links: http://delicious.com/saunier/cv

Software Development

Because I am relying on OpenCV for computer vision functionality, and because C++ is fast, the previous software was written in C++. For the same reasons, the core most computationally intensive functions should be written in C++ (although the python wrappers are more and more usable). The most up to date documentation is at http://opencv.itseez.com.

The platform of choice for development is Linux (e.g. the Ubuntu distribution). I would like to have the code cross-platform, ie the C++ should compile at least under Windows and Linux, which is not too difficult if using the right tools (g++, make or CMake).

I am If you are not familiar with any of the following topic, please read more:

• C++: see below. I recommend the use of smart pointers for ease of memory management (avoiding memory leaks in a program meant to process hours of video without crashing is essential). See Boost shared pointers http://www.boost.org/doc/libs/1_46_1/libs/smart_ptr/smart_ptr.htm
• Version Control: I refuse to work in teams without software version control. I use mercurial which has a fairly low barrier to entry and good documentation. http://mercurial.selenic.com
• Test-driven development: writing tests takes a bit more time when developing, but helps in testing and ensures that the software still matches its specifications when refactoring later. I have used the Boost test library and it does the job http://www.boost.org/doc/libs/1_47_0/libs/test/doc/html/index.html, Google test is getting famous and is used in OpenCV.
• Computer vision algorithms: see above
• Python is nice for visualization, and the binding to OpenCV seem now robust enough for prototyping.

It is very important for me to develop quality software that can be easily maintained over the long term, hence the emphasis on version control, smart pointers and tests.

Traffic Intelligence

Clone with Mercurial from https://bitbucket.org/Nicolas/trafficintelligence/wiki/Home and follow instructions there for installation and use, including OpenCV and the Library for Trajectory Management, providing I/O functions and several trajectory distances (https://bitbucket.org/trajectories/trajectorymanagementandanalysis).

Traffic Intelligence is developed under an open source MIT license and I would like additions to be under the same license.

Cameras

• Vivotek IP8151: used at McGill, issues of framerate at higher resolution
• Panasonic, eg http://www.panasonic.com/business/psna/products-surveillance-monitoring/network-security-cameras/fixed-cameras-color/WV-SP509.aspx
• use portable personal video recorders such as archos (old technology?)
• CMOS sensors, eg http://www.thorlabs.com/NewGroupPage9.cfm?ObjectGroup_ID=2916, http://www.edmundoptics.com/onlinecatalog/Browse.cfm?categoryid=1569

Free Online Datasets

• Old synthetic and traffic video data http://i21www.ira.uka.de/image_sequences/
• MIT car data http://cbcl.mit.edu/software-datasets/CarData.html and person data http://cbcl.mit.edu/software-datasets/PedestrianData.html
• UIUC car detection http://cogcomp.cs.illinois.edu/Data/Car/ and CMU car data http://vasc.ri.cmu.edu/idb/html/car/
• UCSD method for people counting with dataset http://www.svcl.ucsd.edu/projects/peoplecnt/
• PETS datasets http://www.cvg.rdg.ac.uk/slides/pets.html
  • 2009: people tracking with multiple cameras http://www.cvg.rdg.ac.uk/PETS2009/ (http://www.cvg.rdg.ac.uk/PETS2009/a.html)
  • 2001: people and cars ftp://ftp.pets.rdg.ac.uk/pub/PETS2001/
• CityCars and CityPedestrians http://www.psi.toronto.edu/index.php?q=flobject%20analysis
• Gavrila http://www.gavrila.net/Research/Pedestrian_Detection/Daimler_Pedestrian_Benchmark_D/daimler_pedestrian_benchmark_d.html
• INRIA dataset used by N. Dalal (HoG classifiers) http://pascal.inrialpes.fr/data/human/
• ETH datasets http://www.vision.ee.ethz.ch/datasets/index.en.html
• NGSIM dataset: highways and urban corridors taken from multiple cameras on high buildings, with the computed results http://ngsim-community.org/
• The PASCAL Visual Object Classes Homepage contains sets of images of objects of various types, including people, bicycles, cars, etc. http://pascallin.ecs.soton.ac.uk/challenges/VOC/ (see also MIT SUN dataset http://groups.csail.mit.edu/vision/SUN/ and Caltech http://www.vision.caltech.edu/Image_Datasets/Caltech256/, MIT CBCL StreetScenes http://cbcl.mit.edu/software-datasets/streetscenes/)
• KITTI vision benchmark suite (images+lidar) http://www.cvlibs.net/datasets/kitti/ (object detection benchmark http://www.cvlibs.net/datasets/kitti/eval_object.php) and Karlsruhe objects http://www.cvlibs.net/datasets/karlsruhe_objects.html

Image datasets of known objects are useful to train and test object classifiers

Resources

• Open source computer vision projects
  • OpenCV http://opencv.willowgarage.com/wiki/
  • http://code.google.com/p/opencv-feature-tracker/ (Warning: many bugs, not a good basis to build upon)
  • https://bitbucket.org/Nicolas/trafficintelligence
  • https://bitbucket.org/trajectories/trajectorymanagementandanalysis
• Version Control http://mercurial.selenic.com/
• C++
  • http://en.cppreference.com/w/cpp
  • http://www.parashift.com/c++-faq-lite/
  • C++ code samples (Boost, OpenCV, etc) http://programmingexamples.net/