|The current method of video scoring is labor-intensive since no autonomous tracking capability exists. After a drop, during which two to six fixed-zoom ground cameras record the flight, each video is manually “read” for payload position in the field of view. This is accomplished frame by frame, with the video reader clicking on a pixel that represents the visual “centroid” of the payload. Each video frame has a bar code with the azimuth, elevation and UTC stamp, so the data for each frame is stored automatically as the pixel is clicked. After the videos are read, the data is processed to determine the position at each frame during the drop. The payload position is then numerically differentiated to calculate velocity. The automated capability of accurately acquiring TSPI is supposed to hasten the processing of each video by autonomously tracking the payload once initialized. Then data from several cameras can be ‘fused’ together to obtain TSPI (of each object in cameras’ field of view).|
The development of such autonomous capability (TSPI retrieving system) should obviously address the following two independent problems. First, it should process video data itself with the goal of obtaining the frame coordinates of a certain point of the test article, say payload’s geometric center. Second, it should provide the most accurate solution of the position estimation problem assuming that information from two or more cameras situated around DZ is available.
|The appropriate software to solve each of two problems has been developed and successfully tested in computer simulations and with the real drop data. The video data processing aims at preparing all necessary data for the following position estimation. It is based on a piece of software developed by PerceptiVU, Inc., namely PerceptiVU Tracker to retrieve x-/y-offsets of the center of the test item from each frame. The PerceptiVU Tracker window with several drop-down option windows is shown in Fig.C. For the batch processing a user encloses the test article to track into the appropriate-size box on the very first frame and then this item is being tracked automatically.|
Figure C. Setting the options for the PerceptiVU Tracker.
|Figure D represents an example of correcting video data (eliminating the corrupted lines, excluding spikes, and filling dropouts) using the 128-point Welch FFT window, and an example of the processed data for three KTMs is shown in Fig.E.|
Figure D. Checking/correcting time histories of the x-/y- offsets
(click here to animate the image).
Figure E. Example of conditioned data for three cameras.
|Processing video data results in conditioned (corrected if necessary and synchronized for all KTMs) Az/El and x-/y-offsets data ready to be passed to a position estimation algorithm that allows a position estimation per se. Mathematically, the problem of the payload position estimation can be formulated as follows: we need to find the three-dimensional position of the payload’s centroid, when its projection , i=1,…,N onto the image plane of N cameras is available. The positions ) of each camera in the local tangent plane (LTP), as well as their focal length, fi, azimuth, , and elevation, , are known.|
|Obviously, each camera contributes with two nonlinear equations (written for the horizontal and vertical planes). That is where it follows from that to resolve for three components of the vector we need to have at least two cameras (N≥2). The more cameras record the drop and more evenly they are distributed, the more accurate position estimate we could get.|
In this project, this problem was cast as a multivariable optimization problem: choose components of the vector that minimize some compound functional. This problem is solved at each instant of time within a Simulink model that employs a standard MATLAB fminsearch function for unconstrained non-linear minimization (based on the non-gradient Nelder-Mead simplex method). The developed software proved to be very reliable and quite robust in handling both emulated and real drop data. Figure F shows the results of processing data shown in Fig.E. It also shows the results of the descent rate estimate. For this particular drop the GPS/IMU data was also available, so the comparison with an exact data demonstrated a perfect match (with an average error of less than 2m that corresponds to the cameras’ pixel size).
Figure F. Position and descent rate estimation using data of Fig.E.
Figures G and H demonstrate another example when data from six cameras were available. The top-left plot in Fig.G presents a bird’s eye view of the KTM constellation around the release point (radial lines show the direction to the test article at the release (solid lines) and impact points (dashed lines)). The bottom plot depicts the Az/El data for all six cameras. The top right plot demonstrates the estimate of the geometric dilution of precision (GDOP), i.e. the maximum accuracy that could be possibly achieved (based on the cameras’ pixel resolution). Figure H shows the result of processing this six-camera data.
Figure G. Results of video data processing for the six-camera cargo drop.
Figure H. Position and descent rate estimation for the cargo airdrop presented in Fig.G.