Proceedings of the Institute of Acoustics

 

Element-level image renavigation for coherent change detection

 

A. Keith, Raytheon Technologies, Portsmouth, Rhode Island, USA
K. Bongiovanni, Raytheon Technologies, Portsmouth, Rhode Island, USA
A. Wilby, Raytheon Technologies, Portsmouth, Rhode Island, USA
J. King, Naval Surface Warfare Center, Panama City, Florida, USA
D. Sternlicht, Naval Surface Warfare Center, Panama City, Florida, USA

 

1 INTRODUCTION

 

Coherent change detection (CCD) aligns a repeat-pass synthetic aperture sonar (SAS) image with a reference SAS image to detect changes between the two images unrelated to navigational errors. CCD was first developed with radar data, making it more advanced in the radar domain. CCD in both radar and sonar domains can provide critical environmental or military information. In the undersea environment, inaccurate navigation makes performing registration and therefore change detection difficult. The CCD innovation described in this paper leverages an existing CCD code set and SAS data beamformer by using the navigational alignment information calculated from the CCD code to rotate and translate the repeat-pass image on to the reference image’s navigational track. This is carried out at the element-level in the existing beamformer. The novel technique described here has similarities to an approach described in Reference [1]; however, the current work employs innovations for operational efficiency.

 

Current SAS CCD approaches use navigation data from already beamformed images to account for navigational differences between reference and repeat-pass images, which involves warping the images.² CCD is commonly used for SAS’s counterpart, synthetic aperture radar (SAR), as well, to detect changes in the environment over time. Currently, SAR CCD is performed on a reference and repeat-pass image, and a two-dimensional correlation map is produced by determining the maximum likelihood estimate of reflectivity between a reference and repeat-pass image.³ One presented SAR CCD method involves a multi-pass algorithm using three consecutive SAR images. This method suggests creating CCD images from two of the image pairs and performing CCD on the two CCD images to minimize clutter. Another SAR CCD method that minimizes clutter uses generalized likelihood ratio tests and incorporates noise into the models to account for patches of low SNR (signal-to-noise ratio).⁴ This element-level modification improves the overall alignment of the reference and repeat-pass images, as compared to using solely the beamformed image level modification in the existing CCD algorithm.

 

2 OVERVIEW

 

The purpose of this research is to build a SAS CCD toolset which allows co-registration to be conducted at an element-level rather than image level to improve the robustness of SAS processing to navigational inconsistencies. The algorithm when implemented will compensate for, for example, interferometric effects caused by inconsistent pass altitudes. Element-level processing will also allow us to examine other techniques that declutter CCD images by looking at coherence levels within each of the images. At this stage, we have built an element-level co-registration tool and demonstrated its functionality against existing well-registered datasets.

 

3 REVISING THE PROCESSING LINE

 

3.1 Modification of SAS Processing

 

Through modification and combination of existing CCD and SAS processing algorithms, reference and repeat-pass synthetic aperture sonar images are aligned at the element-level, which means that the images are aligned within the SAS processor during the beamforming process. Instead of aligning the images from different viewpoints with CCD alone, the viewpoints of the images are shifted to compare the same patch of seabed. Figure 1 (a) depicts the flow of a conventional SAS processor, and Figure 1 (b) depicts the flow of a modified SAS processor. When using the conventional SAS processor for high frequency SAS processing, it was observed that the motion compensation stage using monostatic phase center approximation incorrectly accounted for phase at short range. Phase errors at short range caused decorrelation between images. In the conventional SAS processor in Figure 1 (a), the scallop routine, commonly referred to as PCA or Phase Center Approximation, applies a time advance, based on the path difference between a bistatic and monostatic system, to received signals to convert the system from bistatic to monostatic. To correct for these errors, the modified SAS processor in Figure 1 (b) scallops the bistatic transmitters and receivers to monostatic phase centers on the processing line rather than the array. By correcting for navigation errors and re-scalloping the SAS data at element-level as opposed to image-level, more precise bistatic to monostatic conversion can be achieved.

 

Figure 1: SAS Processing Flow Diagram. Diagram (a) represents a conventional SAS processor. Diagram (b) represents a modified SAS processor.

 

Once both the reference and repeat-pass data have been beamformed with the modified SAS processor, the reference and repeat-pass images are passed into the CCD process as shown in Figure 2. CCD has 3 main stages relevant to the modifications: navigational alignment, fine-scale co registration and local (or sub-pixel) co-registration.⁵ Within these three stages, the two images are compared, and navigational alignment and co-registration are performed on the repeat-pass image to align the images. As shown in Figure 2, at the conclusion of each CCD process, a change detection map is produced through Canonical Correlation Analysis (CCA) to allow the user to visually identify any significant changes between the two images. After CCD has been performed on the two images, the rotation and translation values applied to the repeat-pass data track in navigational alignment are used to alter the ‘Scallop(2)’, ‘DPCA(2)’ and ‘Azimuthal Resampling’ functions of the SAS processor to revise the repeat-pass element-level SAS data. The unaltered element-level repeat-pass data is then passed into the revised SAS processor beginning at the ‘Scallop(2)’ function, with the revised scallop function referred to as ‘Scallop(3)’. To compare the effectiveness of the element-level alignment of the images, the reference image and revised repeat-pass images are passed into the CCD process once more, and the results of fine-scale co-registration and local co-registration are compared to the original results.

 

Figure 2: Element-level Image Renavigation Process Diagram. Blue represents SAS beamforming. Orange represents the CCD process.

 

3.2 Realignment of Repeat-pass Image Processing Line

 

To realign the repeat-pass data onto the reference data processing line, the original SAS processor featured in Figure 1 is altered to account for navigational differences found in CCD, and is used to re-process the repeat-pass image. The data used in this paper uses the Office of Naval Research Synthetic Aperture Sonar of Reference [4]. The processing line is a characteristic of the ω-k beamformer versus a back projection beamformer where the navigation path is more directly applied in the motion solution. Figure 3 depicts the reference and repeat-pass processing lines with no revisions made in ‘North, East, Down’ (NED) coordinates. Since the reference and repeat-pass images were collected during separate vehicle runs, navigational differences exist between them, and this is apparent by their differing processing lines. This indicates that the view of the seabed for each image differs, and this makes CCD more difficult to perform. Thus, by rotating and translating the repeat-pass processing line to align with the reference processing line, the images will have a view of the same patch of seabed, making it easier to detect significant differences between the two images, while eliminating warping within CCD.

 

 

Figure 3: Original Reference and Repeat-pass Processing Lines

 

After the reference and repeat-pass images are beamformed by the SAS processor, they are input into the CCD process. CCD performs a series of shifts on the repeat-pass image to align it with the reference image. The first stage in CCD is called “Navigational Alignment”. This function compares the reference and repeat-pass navigational tracks in the NED coordinate space and calculates the rotation and translation values to apply to the repeat-pass image to align it with the reference track in the pixel coordinate system. To revise the repeat-pass processing line, these rotation and translation values are calculated in the NED coordinate system. The rotation value is calculated using right triangle geometry by taking the inverse cosine of the reference unit vector divided by the repeat-pass unit vector. The translation value is calculated by choosing corresponding points on the reference and repeat-pass tracks and using the Pythagorean distance formula to calculate the distance between the tracks. These rotation and translation values are then input into the revised SAS processor.

 

Since the repeat-pass image is the only image being revised at the element-level, the reference SAS data remains untouched by the revised SAS processor. To revise the processing line, the only functions that are altered are ‘Scallop’, ‘DPCA’ and ‘Azimuthal Resampling’. Thus, this revised SAS processor takes in the repeat-pass data through ‘DPCA(2)’ and continues with a third scallop function, ‘Scallop(3)’, until the end of the process. To project the repeat-pass phase centers onto the reference processing line, the calculated slant range by which this function scallops the data is altered to take these rotation and translation values into account. The adjusted slant range becomes 𝑑= 1/2(𝑑_{𝑟𝑜𝑡𝑎𝑡𝑖𝑜𝑛} + 𝑑_{𝑡𝑟𝑎𝑛𝑠𝑙𝑎𝑡𝑖𝑜𝑛} + 2 ∗𝑂𝑓𝑓𝑠𝑒𝑡 𝐷𝑖𝑠𝑡𝑎𝑛𝑐𝑒) , where the offset distance is equal to the sum of the translation and rotation distances or 𝑑_{𝑡𝑟𝑎𝑛𝑠𝑙𝑎𝑡𝑖𝑜𝑛} + 𝑑_{𝑟𝑜𝑡𝑎𝑡𝑖𝑜𝑛}. These translation and rotation distances are depicted in Figure 4. The translation distance is simply the translation in meters of the processing line given by CCD, as given by 𝑑_{𝑡𝑟𝑎𝑛𝑠𝑙𝑎𝑡𝑖𝑜𝑛} in the top diagram. The rotation distance, 𝑑_{𝑟𝑜𝑡𝑎𝑡𝑖𝑜𝑛} must be calculated using right triangle geometry to project the repeat-pass processing line onto the reference processing line, as shown by the bottom diagram in Figure 4. As depicted in the diagram, the rotation distance is equivalent to the equation 𝑑_{𝑟𝑜𝑡𝑎𝑡𝑖𝑜𝑛} = 𝑎𝑑𝑗𝑎𝑐𝑒𝑛𝑡∗𝑡𝑎𝑛⁻¹ (𝜃) , which is calculated for each ping.

 

 

Figure 4: Applying Navigational Alignment to Scallop Function

 

Since the ‘Scallop (3)’ routine has been adjusted to incorporate the translation and rotation navigational alignment values, this must be incorporated into the DPCA function or ‘DPCA (3)’ which provides motion compensation information vital to the azimuthal resampling routine. In DPCA, a heading value is assigned to the pings, and since the heading of the pings has been rotated to account for the rotation of the repeat-pass processing line, this heading value is edited to add in this rotation angle.

 

3.3 Validating Beamformed Images

 

The revised repeat-pass image is initially validated by comparing the original reference and repeat pass images to the revised repeat-pass image before it is passed through CCD (See Figure 5). By comparing the location of prominent seabed features, such as a small crater, in each image, it is clear that the images are not perfectly aligned in either the original or revised repeat-pass images, but this is fixed through the CCD process. However, the rotation of the repeat-pass processing line onto the reference processing line is nearly perfect. In Figure 5, a red line begins at the aforementioned small feature in the reference image and extends out to the middle of a prominent small crater. This line was copy and pasted onto both the original and revised repeat-pass images, starting at the same small feature as the reference image. If the images have the same rotation, this line will start and end at the same features as the reference image. With the original repeat-pass image, the red line starts at the same feature as the reference image, but the arrow points notably above the small crater that the line in the reference image points directly to. This indicates, as expected, that the rotation of the original repeat-pass image is not equivalent to the rotation of the reference image. In contrast, the red line begins at the same small feature in the revised repeat-pass image and ends at the same location as that of the reference image. This demonstrates that the repeat-pass processing line has been successfully rotated on to the reference processing line. Therefore, using the CCD navigational alignment rotation information, the repeat-pass SAS data can be re-processed at the element-level to rotationally align with the reference image to improve change detection.

 

Figure 5: Fine-scale Co-registration of the Original versus Revised Repeat-pass Image

 

To further validate the repeat-pass image beamformed using the revised processing line, the reference and revised repeat-pass images are passed through CCD and the CCD results of the images are compared before and after revision. Since the navigational alignment step is performed at the element-level, element-level renavigation negates this stage of CCD. The following stage of CCD is fine-scale co-registration, which applies a scalar horizontal and vertical shift value to all the repeat-pass image pixels. Since this translation is not accounted for at the element level, this stage of CCD behaves as it does with the unrevised repeat-pass image and shifts the repeat-pass image within pixels of the reference image using the scale-invariant feature transform (SIFT) algorithm. After fine-scale co-registration, the reference and fine-scale co-registered revised repeat-pass image are passed into the local co-registration stage of CCD. Local co-registration computes navigational errors, such as rotation, heave, surge and sway, between the reference and repeat-pass image and co registers the images to improve change detection between the images.

 

Using the navigational errors, along-track and across-track translation estimates are computed for the repeat-pass image to translate it at a higher precision than the fine-scale co-registration to align it with the reference image. Figure 6 presents the along-track translation estimates made within the local co-registration stage of CCD for the original repeat-pass image (on the left) and the element-level revised repeat-pass image (on the right). The amount of translation in pixels at each pixel in the image is represented by the color bar, ranging from -5 pixels (dark blue) to 5 pixels (dark red). For the original repeat-pass image, there is a wide range of along-track pixel translation estimates, denoted by the rainbow of color throughout the image map. This indicates that there is a wide range of differences between the reference and original repeat-pass images, from little difference (light green) to a large difference (dark blue and red). For the revised repeat-pass image, the along-track translation map is relatively uniform aside from the parts of the image close to the track and far away from the track, which is to be expected. The majority of the pixel translation map is zero translation (light green) or between -1 and 1 pixel translations (light blue to light yellow). As demonstrated with the fine-scale co registration results, revising the repeat-pass image at the processing level removes significant navigational discrepancies between the reference and repeat-pass SAS data. Thus, this improves the change detection process by helping CCD to align the images more accurately with less navigational differences to overcome, as shown with the local co-registration along-track translational estimates in Figure 6.

 

Figure 6: Comparing Translation Estimates of Original and Revised Repeat-pass Images

 

4 CONCLUSION

 

Through several modifications, the existing coherent change detection (CCD) process has been improved. The major modification outlined in this paper involved taking the rotation and translation values calculated by navigational alignment in CCD, and, utilizing a fast ω-k SAS beamformer, using it to rotate and translate the repeat-pass image onto the reference image’s processing line. The SAS processor was modified to incorporate the navigational alignment rotation and translation information. The reference and repeat-pass images were run through the CCD algorithm before and after the element-level repeat-pass data revision and compared at each stage. Through comparing the results, the revised repeat-pass data was aligned nearly perfectly to the reference data, while the untouched repeat-pass data was aligned closely with the reference image but not as closely as the revised repeat-pass image.

 

The authors’ objective is to implement precision CCD co-registration for real-time application. This research differs from the approach of Reference [1] in that it operates on the linear processing line of an (inherently) fast ω-k SAS beamformer; whereas, [1] renavigates the non-linear tracks from a back projection beamformer – an image formation process that is computationally more expensive. Furthermore, the approach described here requires considerably fewer re-beamforming iterations than what is reported in Reference [1].

 

Overall, a toolset has been built which re-registers by running an initial image-based co-registration algorithm, reverting to element-level data and manipulating individual elements prior to image formation – thus, eliminating the need for warping and rotating. This process has demonstrated consistent sub-pixel level registration across the swath.

 

5 FUTURE WORK

 

With this improved method of processing and change detection for SAS images, there is a plethora of future capabilities that can build upon this progress. One such possibility is to test this element level renavigation approach on more challenging scenarios. The tracks of the datasets used in this paper are relatively consistent between the reference and repeat-pass images. Evaluation of the element-level adjustment approach for robustness and improvement will rely on exploiting datasets with greater track offset.

 

Another area of research is sub-swath coherence-based CCD decluttering. Reference [4] describes a process used in SAR to declutter CCD images by looking for what is described as Non Return Areas (NREs). These are used to generate a mask to declutter the CCD image. In SAS, we have region of shadow, which are these NREs, and also regions of rapid movement. The CCD process proposed in this paper reverts to element level data and so synthetic sub-apertures could be used to identify NREs and other regions with poor coherence.

 

The precise reformation of repeat pass images, without the need for subsequent rotation and warping, provides highly aligned results which are suitable for use in CCD, as well as incoherent and target based change detection approaches. The modified scalloping algorithm is efficient (not computationally intensive) and only applied to repeat pass data. Additional future work includes the quantification of performance improvements and limitations through analysis of both field and simulated data, as well as determining/optimizing computational timing and sizing for incorporation into real-time operation.

 

6 ACKNOWLEDGMENT

 

This coherent change detection (CCD) project is sponsored by the U.S. Office of Naval Research (ONR), with SAS data provided by the Naval Surface Warfare Center (NSWC) Panama City.

 

Unrestricted Content. “This document does not contain technology or Technical Data controlled under either the U.S. International Traffic in Arms Regulations or the U.S. Export Administration Regulations.”

 

7 REFERENCES

 

1. T. O Sæbø, R. E. Hansen, H. J. Callow, and S. A. V. Synnes. “Coregistration of Synthetic Aperture Sonar Images from Repeated Passes.” presented at the Underwater Acoustic Measurements Conference, 2011.
2. D. Sternlicht, J. K. Harbaugh, and M. A. Nelson. “Experiments in Coherent Change Detection for Synthetic Aperture Sonar.” Proceedings MTS/IEEE OCEANS, pp. 1–5, October 2009.
3. M. A. Richards, "A Beginner's Guide to Interferometric SAR Concepts and Signal Processing [AESS Tutorial IV]," in IEEE Aerospace and Electronic Systems Magazine, vol. 22, no. 9, pp. 5-29, Sept. 2007, doi: 10.1109/MAES.2007.4350281.
4. M. Newey, J. Barber, G. Benitz and S. Kogon, "False alarm mitigation techniques for SAR CCD," 2013 IEEE Radar Conference (RadarCon13), Ottawa, ON, Canada, 2013, pp. 1-6, doi: 10.1109/RADAR.2013.6586144.
5. T. G-Michael, B. Marchand, J. D. Tucker, T. M. Marston, D. D. Sternlicht and M. R. Azimi Sadjadi, "Image-Based Automated Change Detection for Synthetic Aperture Sonar by Multistage Coregistration and Canonical Correlation Analysis," in IEEE Journal of Oceanic Engineering, vol. 41, no. 3, pp. 592-612, July 2016, doi: 10.1109/JOE.2015.2465631.