Master’s Project Defense
1 / 31

Master’s Project Defense Bill Champlin University of Colorado at Colorado Springs Dept of Computer Science - PowerPoint PPT Presentation

  • Uploaded on

Master’s Project Defense Bill Champlin University of Colorado at Colorado Springs Dept of Computer Science. Java Quasi-Connected Components (JQCC) Tracking System April15, 2010 Advisor - Dr. Terrance Boult. Agenda. Overview/Scope Approach Activities Analysis & Design Re-host Effort

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Master’s Project Defense Bill Champlin University of Colorado at Colorado Springs Dept of Computer Science' - manchu

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Master’s Project DefenseBill ChamplinUniversity of Colorado at Colorado SpringsDept of Computer Science

Java Quasi-Connected Components (JQCC) Tracking System

April15, 2010

Advisor - Dr. Terrance Boult


  • Overview/Scope

  • Approach

  • Activities

  • Analysis & Design

  • Re-host Effort

  • Port to Linux

  • Effort &Performance Metrics

  • Adapting QCC to a new Domain

  • Lessons Learned/Issues/Recommendations

  • JQCC Demonstration

Project overview
Project Overview

  • JQCC re-hosts Quasi-Connected Components (QCC) portion of the Lehigh Omni-Directional Tracking System (LOTS) to Java

    • Object tracking software used to track snipers, ships, UAVs, etc

  • Dr. Boult wanted a version to post on the web

    • Means Java for maximum portability

    • Need to prove that by running different platforms

    • Would Java be fast enough? i.e. 30 FPS?


  • Options: write from scratch or port QCC portions of LOTS QTRACK?

    • From scratch, had these obstacles:

      • Had no track region or “cluster” association algorithm

      • Legacy had numerous special cases to handle tracking issues - would be missing those

    • Porting LOTS QTRACK daunting

      • QTRACK baseline ~ 96 KSLOC of C++

      • “You won’t understand what a lot of what the code does” – Dr. Boult

      • Complex, undocumented and sparsely commented

        • Supports various input sources, architectures i.e. Intel MMX and camera types

        • A lot of dead code

  • Chose to port QTRACK

    • Knew it worked – just had to translate?

    • Not quite – a lot of architecture work first


  • Obtain LOTS s/w baseline (QTRACK) and get running

  • Extract core functionality (QCC)

  • Develop requirements and UML design

  • Port LOTS core functionality to Java (JQCC):

  • Verify execution under Windows and Linux platforms

  • Evaluate performance vs. legacy QTRACK

  • Adapt to tracking night sky images

  • Generate promised “artifacts”

    • SRS, Certification Specification, and User Handbook

    • Project Report

    • Project Website:

Analysis model
Analysis Model

  • Requirements broken out by JQCC Use Cases:

Design model
Design Model

Concurrency diagram depicts top level process/thread allocation

Qcc technique from 20k ft
QCC Technique (from 20K ft)

QCC tracks targets by:

  • Building a background model

    • Blends input frames into primary and secondary “reference” images

      ref = (1-alpha)ref + alpha*ref // big alpha blends faster

      • Primary ref = ref closest to input frame pixel and will be used for thresholding – means a more accurate threshold

    • Thresholding / Connecting Thresholded Pixels

      • Identifies candidate target pixels exceeding dynamic high threshold

        • Sum them into a 4x4 reduced “Parent” image

      • Identify neighbors that exceed a lower threshold around target pixel

        • Sum into into Parent image (form small blobs or “regions”)

      • Adjust dynamic threshold for next time based on pixel count exceeding high threshold

Qcc technique con t
QCC Technique-Con’t

  • Parent regions are grown and “connected” to those from last parent - get target ID (Label) from past parent if they overlap a region from last time

  • All pixels of the same region are “labeled” with the same target id

  • Regions are matched with previous overlapping and non-overlapping regions

  • Regions that are too sparse or to big (due to lighting shadows/etc) are “cleaned” (discarded)

  • Regions are assigned a “confidence” based on density, shape, speed, etc and classified “tracks” if they exceed confidence threshold

  • Every nth frame, reference and variance frames are updated

Qcc performance boosters
QCC Performance Boosters

  • Thresholding, swapping references done over column segments of each row (cords)

  • Reference frames only blended every 16th frame or so (configurable)

    • Input buffer “absorbs” periodic slow downs

  • Parent image reduces pixels to process

  • Only parent rows with used pixels are processed

Porting to java approach
Porting to Java Approach

  • Keep legacy track class structure where possible

  • Preprocessor statements conversion

    • #defines variables to static final or static variables

    • Groups of related #defines converted to Java enums

    • #define macros to functions

    • #if, #else, #endif converted to if, if-else blocks

      • Had to put debug in QTRACK to figure out what defines were “TRUE”

Porting to java approach con t
Porting to Java Approach – Con’t

  • Convert unsigned char to byte, etc.

    • Save memory over unsigned char to short or int

    • No unsigned primitive types in Java – use masks

      • Example: ucharval=0x8F;int result = 0xFF – val;

        Convert to: byte val=0x8F;int result = 0xFF – val;

      • Error: want - 0x70, get - 0x170

        Java converts byte val to -113, then subtracts – logic error

      • Instead use: byte val=0x8F;int result=0xFF-val&0xFF;

      • Error prone – QTRACK uses a lot of compound expressions

    • Java right shift “>>” is a “signed” shift, shifts in 1’s if value being shifted has sign bit set

      • Use “>>>” instead to shift in 0’s from left to match C++

Porting to java approach con t1
Porting to Java Approach – Con’t

  • No pass by reference “&” in Java – only by value

    • Convert reference or “out” arguments to “wrappers”

      public class ByteWrapper{ public byte val;}

      • Caller sets val , method gets val inside, method sets val before returning, caller retrieves val after method returns

      • Autobox types i.e. Integer, Double, etc. don’t work for this

  • No pointer arithmetic in Java

    • QTRACK uses pointer arithmetic extensively

    • Convert pointers to use indexed arrays and do math on the indexes

      • Error prone separating ptr math from “content” math

  • Convert static function variables to static class fields

  • Port to linux
    Port to Linux

    • Goal – a portable, ready-to-run solution that doesn’t force the user to download and install a lot of packages (preferably nothing but Java)

    • Biggest issue – getting movies played/injected on Linux using Java

    • JMF pure Java version advertised as runnable under Linux

      • Unable to run JMF MPEG Camera Simulator under RedHat Enterprise 5.0

    • Examined other options:

      • XuggleXuggler (licensing issues / involved to build source on Linux)

      • FOBS4JMF (requires FFMPEG, other C libs)

    • Finally used Java Advanced Imaging (JAI) to load JPEG image sequences

      • Created a 2nd version of the Camera Simulator (JPEGCameraSimulator)

    • Same JQCC class files now run under Linux

      • Startup bash scripts only difference – can run same Java class files as MS Windows

    • Verified under both Ubuntu 9.10 and RedHat Enterprise 5.0

    Adapting jqcc to new domain
    Adapting JQCC to new Domain

    • Domain chosen: Tracking objects in night sky images

    • Difficulty finding sample movies

      • Web searches not that productive

    • NASA JPL Search

      • Very short videos of Martian moons Phobos & Deimos

      • GALEX Space Telescope Debris Movie [5]

    • Ran across Kevin Fetter’s web site[3]

      • Movie camera telescope setup

      • Had no images when I contacted him

      • Collected several sets of satellites for me moving at various directions and speeds (GEO, Near Earth, Blinker) (has collected a lot more since then)

    • Initially, JQCC would track only one of the fast satellites

    • Modified parameters to get it to work for most images

    Lessons learned
    Lessons Learned

    • Java weaknesses

      • Movie support

      • No unsigned types

      • Garbage Coll. causes display updates to “pause”

        • Can’t avoid it totally – JVM / open source packages

        • GC runs longer the more memory there is allocated

        • Work-arounds

          • Use incremental “-Xinc” option to help GC “pauses”

          • Consider using direct memory allocation to allocate memory outside of JVM so GC doesn’t check it

      • Using signed types with masking is error prone

      • Predicates with final constants leads to “dead code”

    • Promise of Java Portability Holds

      • Write once, build once, run anywhere (with a JVM) works

    Issues recommendations

    • Modify JPEG Camera Simulator to “read ahead” instead of loading all frames into memory

    • Find and fix possible dynamic thresholding issue

    • Examine tracking stability

    • Improve reusability - finish supporting all tuning parameters in calibration file and on GUI; remove scenario specific processing

    • Finish debugging rectangle files for “ignoring” regions and outside of a radius for omni-directional images

    • Re-factor into an OO design; add more / better comments

    • Improve display performance / fix “update” color artifacts

    • Implement remaining capabilities

    Demonstration script satellite images courtesy of 4 galex debris courtesy of 5
    Demonstration ScriptSatellite images courtesy of [4], GALEX Debris courtesy of [5]

    • Low earth satellite

      • Rate Adjustment / Tracking Parameters

    • Vertical Object

    • MPEG GEO

    • Blinker

    • Linux low earth satellite

      • Side by side comparison

      • Frame Rates

    • GALEX Debris

    • Toolbag


    For every “used” pixel i

    primaryDiff = |inim[i] - refim1[i]|

    secondaryDiff = |inim[i] - refim2[i]| // use closest difference

    if secondaryDiff < primaryDiff

    diff = primaryDiff

    swap(refim1[i], refim2[i]) // keep bkgrnd closest to last input as primary

    thresh = dynThresh + varim[i]

    if diff[i] > thresh // threshold difference frame

    outim[i] = diff


    outim[i] = 0

    //Build parent image (see next slide for what a parent image is)

    For pos in every parent pixel

    For j in every image pixel pos that maps to parent

    if outim[j]


    if outim[i]-delta > lowThresh

    parent[pos] += 256 // Increments count above low threshold

    Differencing con t
    Differencing - Con’t

    • A parent is a smaller image that summarizes regions across the larger FOV

    • Each parent pixel maps to a 4x4 pixel in main image

    • Each parent pixel is a short which counts pixels above high threshold and low threshold at same time

      • Upper 8 bits count of pixels > High threshold

      • Lower 8 bits sums pixels > Low threshold

    • Smaller image = faster processing for subsequent processing i.e. growing regions

    Track processing

    Discards regions that are too small

    Track Processing

    Associates region with past targets that overlap in last update

    See Differencing slide

    Dynamic threshold adjusted up based on count of differenced pixels > threshold

    Associates region with past regions that have gaps between them

    Parent regions grown and matched to overlapping regions from past parent

    Associates region with regions older than last update

    Assigns a common label (integer value) to all parent pixels of a region

    Discards regions that are too big or sparse

    Reducing threshold around target improves tracking

    Regions sorted by confidence for next update comparison

    Confidences converted to priority which dictates next threshold reduction

    Input frame blended into ref. frames and variance frame updated to be slightly bigger than background

    Metrics con t
    Metrics – Con’t

    Metrics con t1
    Metrics – Con’t

    Cocomo ii jqcc effort estimate using costar 4 tool
    COCOMO II JQCC Effort EstimateUsing COSTAR[4] Tool

    More lessons learned
    More Lessons Learned

    • Threading with SMP architectures improves performance

    • Forget about design and code “zealousies”

      • Do what’s simplest when translating code

        • Using Java Enum wasted time over constants

          • Had to create code and convert using code

        • Forget about making object oriented initially

          • Don’t disturb the existing design that works; refactor later


    • [1] T.E. Boult, R.J. Micheals, X. Gao, M. Eckmann, “Into the woods: visual surveillance of non-cooperative and camouflaged targets in complex outdoor settings”, in Proc. Of the IEEE, Oct. 2001

    • [2] T.E. Boult, T. Zhang, R.C. Johnson, “Two threshold are better than one”, CVPR, pp.1-8, 2007 IEEE Conference on Computer Vision and Pattern Recognition

    • [3] K. Fetter, Astronomical Images,

    • [4] Softstar Systems, Costar Software Estimation Tool,

    • [5] GALEX Science Team - California Institute of Technology, GALEX Space Telescope Movie,