CENL: Video Tracker

Overview

ZTracker is a LabView program aimed at tracking a moving object and recording its (X,Y,timestamps) coordinates as fast as the camera allows. It uses any webcam recognized by NI MAX (the National Instrument Automation Explorer, 5.5 or later), which virtually recognizes any modern commercial webcams.

• ZTracker allows the user to enter any event at any time (e.g. 'start the experient now', 'new trial') and this event is timestamped.
• The user can define a 'region of interest' in which tracking will occur (ignoring the other regions).
• The user can choose between 2 tracking algorithms: Intensity detection and Motion detection.
• The tracker reads an arbitrary number of predefined locations from a file (e.g. feeders) and generate a timestamped event each time the location is visited.
• The frame rate can be specified dynamically by the user, and the Tracker will attempt to meet it if possible.
• The user can save all or portions of the raw tracking image as a movie.
• ZTracker connects to the Neuralynx Data acquisition system to share its time stamps and events.
• Our Note taking program 'LabBook' connects to the tracker to share time stamps and generate events as needed.

Details and Quick Guide

Setup:

• To run this program you will need the Labview 2012 runtime engine or later, free from National Instrument, the Vision run time engine 2013 also free from National Instrument. Finally, you will need the Vision Acquisition Software module which may or may not be free for you (you can do a 45 day evaluation period for free). You also need a camera. Note: The frame rate and image adjustment that are possible depend on your camera and ZTracker will adapt to it. We have used/tested a PS3eye USB camera (~50 fps) and regular commercial USB webcams (~20-25fps).
  PS: Here are links to the actual components you need to download, if the links above did not work for you: the 2012 runtime engine (232MB) , the Vision run time engine 2013 (176MB). the Vision Acquisition Software (2.8GB)
• One time camera setup: After you install all the software pieces above (you need to reboot the computer), plug your camera, find the NI MAX icon on the desktop. Open it. and open the 'Devices and Interfaces' tab. You should see your camera listed there. Note its designation (e.g. cam0). Select it and hit 'Grab' to make sure it works. Go to the 'Acquisition Attributes' tab at the bottom of the screen, and select 640x480 RGB24 (30 fps) (you can change this later to test what works best for you). Then click 'Save' and close NI MAX.
• Create a folder where ZTracker target locations will be stored. Edit the ZTracker.ini file. An example of target location file is provided (pixel coordinates, red dots on the left panel in the figure below). This file can be generated by hand (trial and error!) or using the Calibration.exe program. Note: You might not need to specify targets, in which case just enter one line containing '0 0'.
• In the ZTracker.ini file update the designation of your camera (given to you above by NI MAX. Typically 'cam0' or 'cam1'...).
• It is recommended that you create a new folder each time you plan on acquiring data (e.g. .../ratdesignation/day/TrackData : '.../Ratatouille/010415/TrackData'). The first time you save the data, ZTracker will ask you for it.

Operation: The figure above shows you the main screen.

• D1 is the tracker display. It shows the last N recorded positions, where N is the trail length. N can be adjusted by the user (default 200). The trail can be erased at anytime by clicking on the 'Erase Trail' button. At the time of this release, the image size has to be standard 640x480, RGB24 (to be set in NI MAX). D1 also shows the target positions read from the file specified in Tracker.ini (red dots). D2 is the raw image captured by ZTracker (but see the 'show preprocessed' button below).
• Panel I allows the user to change some parameter settings of the camera. Those settings are camera dependent (e.g. brightness, contrast...).
• The desired frame rate (defaut 25) can be changed at any time by the user. The actual frame rate is displayed quasi instantaneously in the 'Rate' yellow meter above D1.
• The label in C is the active camera. It is read directly from the Tracker.ini file which can be edited by the user. The computer can have several valid webcams, but only one can be specified when ZSTracker starts. Other programs (e.g. VideoRecord.exe, LabBook.exe) can be run in parallel and use the other webcams if needed.
• The user can change the Resolution at any time. 1 is high resolution (divides the picture in 80x80 squares), 2 is lower resolution (40x40). Adjust the resolution to control for tracking noise (jitter) and frame rate.
• The Connect to Cheetah button connects the tracker to the Cheetah data acquisition system. When connected, the time stamps are imported from Cheetah (displayed in T), and any event entered in the Tracker (see below) are also sent to Cheetah. By default the Cheetah server (Serv) is the machine on which ZTracker is run (localhost). Enter a valid IP address before connecting if you are running Cheetah on a different computer. Note: for safety reasons (time stamp integrity), you cannot disconnect, unless you quit ZTracker.
• The 3 paths in P show the track file (top, textfile containing (t,x,y)), the Event file (middle, textfile) containing the time stamped event entries, and the eventual movie file (if created, bottom, .AVI). The event and movie file names are automatically contructed from the track file. ZTracker will automatically ask you for the track file name the first time you flip the 'Save' switch.
• The blue panel, under the timestamp (T) contains an indicator that turns green when the coordinates are within X points (in Range) from one of the targets (red dots in D1). X is specified in 'Radius' (in pixels, default 35). The 'Now at' indicator, in the panel underneath, indicates the target where the animal currently is (-1 if at no target, or the target number). There is a way of dynamically asking ZTracker to detect the arrival at a specific target ('Target') that can be specified at any time by other programs (COM connections... this is advanced.. ask us about it if needed !). Wrong flashes green if the animal reaches a target that is not the one specified. Inactive flashes if the tracking does not move more than Radius pixels, for the entire duration of the trail length. These events are automatically time-stamped and recorded in the Event file.
• 'Jump Control' is a switch that prevents the tracker from jumping from one location to the next (1 frame) if that location is more than Radius pixels away from the current location. Use this to reduce/improve tracker noise.
• 'Algorithm' specifies the detection algrithm used. At the time of this writing there are 2 choices. Intensity Detection: The trackers follows the brightest (whitest) point, from frame to frame. Use this is the environment is dark and the animals carries a LED for example. Motion Detection is not sensitive to light levels and follows the area of maximal changes from frame to frame
• The Save switch will save the data (track and event files)
• To the right of D2, you will find a ROI definition panel. You can select a rectangle, tilted rectangle, polygone, arbitrary curve, circle or donut-arc. Once selected, click in D1 and arange the size/orientation/position of the ROI. If the toggle R 'Restrict to ROI' is off, an event will be generated each time the tracking enters the ROI. If the toggle in on, then no event is generated, but all tracking will occur within the ROI (i.e. anything outside will be ignored).
• The Show Preprocessed toggle will switch D1 between the raw data, and the data actually used after all preprocessing (ROI, smoothing, thresholding etc...). Use this to assess the extent to which the options you have chosen are adequate, and to find the best set of options for your setup.
• Smoothing kernel will let you add a 2D smoothing kernel to the image before it is assessed by the Detection algorithm. Use this to decrease noise.
• Thresholding contain an instantaneous display of the current detection value. Any value below the threshold will be ignored, and the tracking will remain still for that frame. Use this to decrease noise.
• The Save movie panel will let you save the raw image in an AVI file (no sound). The movie can be full size (640x480) or half size (320x240). The compressor is set as soon as you flip the switch (so set it before you start recording). The default is the standard DV Video Encoder.
• Below D2, you will find the event entry area. At any time write an event (e.g. 'start trial') and click the red button. The event is timestamped at the time of the click, and saved in the event file. The last event recorded is indicated to the right of the red button.
• Notes: Advanced features (using the COM Windows features)
  • The user can specify one of the targets as the 'next target' dynamically (i.e. from another program). An event is generated when it is reached. This event can be sent to an external program (e.g stimulator, or behavioral apparatus control)
  • Other programs can query ZTracker for its time stamp, or send ZTracker an arbitrary event to store in its enevt file.
  • In principle, any ZTracker feature (e.g. switches on front panel) can be controlled by an external program. The (x,y) coordinates can be read from ZTracker in real time (e.g. by external behavioral programs).

We develop this program as needed and welcome comments and suggestions!

We grafully acknowledge NSF grant: 1117303 and ONR MURI N000141310672.

Brought to you by the Computational and Experimental Neuroscience Laboratory