Extracting Ground Truth from Surveillance Video

In the early days of the ADAPD project, I served as the "data czar" and helped acquire and organize a large amount of data about a series of underground explosive tests that took place in Nevada during the DAG phase of the Source Physics Experiments venture. While we had high-resolution seismic data from multiple sensors, we did not have much insight into what was going on any particular day. Fortunately, the DAG researchers had maintained a date-stamped webcam for the worksite that captured a distant view of the site every 10 seconds. After a lot of gritty data engineering work, we were able to extract a good bit of ground truth from the video to help explain what was going on in the seismic data. We wrote the below report in FY20 to cover all the details, but had to wait for a long embargo period to expire before we could release it publicly.

There are a few interesting things in this report:

• To verify the webcam's clock was correct, we plugged the camera's coordinates into Google Earth and compared sunrises.
• I wrote an OpenCV script that did some ad hoc image processing to extract the date stamp from the bottom of each image.
• We manually boxed over 2,200 image samples for 18 types of vehicles.
• I used file timestamps to measure how my own annotation rate improved from 5 objects/minute at the start to as many as 17 objets/minute once I had updated the labeling tool.

Abstract

The Advanced Data Analysis for Proliferation Detection (ADAPD) project is a NNSA NA-22-sponsored Venture that is developing novel data analysis capabilities to detect low-profile nuclear proliferation activities. A key step in the information refinement process for this work is to inspect input sensor datasets and data products produced by our analytics to generate as much "ground truth" as possible about the events that took place during the period of observation. This information helps the team's data scientists improve and validate their algorithms and yields data products that are valuable to analysts and decision makers. In this report we provide information about how we inspected multimodal sensor data from the Source Physics Experiment's Dry Alluvium Geology (DAG) tests and generated ground truth for ADAPD's analysis teams. This work illustrates the front-end data engineering tasks that frequently arise in new studies and documents our efforts to gain greater confidence in the assessments of the data.

Publication

• SAND Report Craig Ulmer and Nicole McMahon, "Extracting Ground Truth from Surveillance Video in the Dry Alluvium Geology (DAG) Experiment". SAND2023-14455, September 2023.