Imaging sensors operating in infrared (IR) region of electromagnetic spectrum are gaining importance in airborne automatic target recognition (ATR) applications due to their passive nature of operation. IR imaging sensors exploit the unintended IR radiation emitted by the targets of interest for detection. The ATR systems based on the passive IR imaging sensors employ a set of signal processing algorithms for processing the image information in real-time. The real-time execution of signal processing algorithms provides the sufficient reaction time to the platform carrying ATR system to react upon the target of interest. These set of algorithms include detection, tracking, and classification of low-contrast, small sized-targets. Paper explained a signal processing framework developed to detect and track moving point targets from the acquired IR image sequences in real-time.

Keywords:   Automatic target recognition, infrared, point target, image processing, detection, tracking

The main functionality of infrared (IR) imaging-based airborne automatic target recognition (ATR) systems is to detect, track, and identify the threats by processing the image information. These ATR systems employ wide field of view imaging sensors operating in MWIR (3 µm - 5 µm) or LWIR (8 µm - 12 µm) regions of the electromagnetic spectrum to exploit the IR scene surrounded by platform. The target is expected to be a point size (pixel size) in the image plane due to the wide field of view of imaging sensors. The less spatial extent (point, small) nature of the target pixels relative to the background pixels makes the signal processing algorithm design more challenging for these applications. The specifications of ATR system is highly demanding in terms of probability of detection and false alarm rate. This calls for highly sophisticated signal processing algorithms. Figure 1, depicts the signal processing data flow in a generic imaging-based ATR system^1,2. The important stages in signal processing data flow include: Image pre-processing, clutter-rejection, tracking and association, and discrimination.

1.1 Image Pre-processing

The pre-processing handles problems related with the IR Imagery. The IR Imagery generally consists of significant level of noise. The noise is usually associated with the non-uniform response of the detector elements. The non-uniformities can be handled to a certain extent by calibrating each detector element. The problem still exists in handling the pixels, either dead (no response) or saturated (excess photo-current). The localised non-uniformities are within one or two pixels in size. The target size in the image is expected to be within a pixel. The elimination of non-uniformities can improve the system performance. There exists a requirement to identify these pixels and to eliminate them for further processing. In recent developments, the imaging sensors are made to realise the image pre-processing as part of sensor rather than part of ATR system.

1.2 Clutter-Rejection

The input to clutter-rejection phase of algorithm is the pre-processed raw image to reject clutter (unwanted signals originated from background). The output image after clutter- rejection stage should ideally contain possible targets. The main goal of clutter-rejection algorithm is to estimate the background and suppress the background clutter level to zero at the same time preserving the signal level. Clutter-rejection (CR) stage operates on every image frame and generates set of detections.

1.3 Tracking and Association

The tracking and association (TA) phase operates on the detections provided by the clutter-rejection module for grouping the detections originated from the same spatial source into single track. TA phase carries out; track initiation, track association, and track-smoothing functionalities.

1.4 Discrimination

Discrimination phase of the algorithm operates on the track database created by tracking and association module. The function of the discrimination module is to identify the track whether it belongs to true target or a false target.

The current work focuses on an algorithm framework for real-time detection and tracking of target of interest with minimal information available related to the target. The paper discusses a clutter-rejection, and tracking algorithm designed for detection and tracking moving point targets from IR image sequences.

Clutter-rejection (CR) stage processes large amount of pixel data from the imaging sensor as a frame to identify few pixels that might contain targets. Clutter-rejection stage has to process the image frames at sensor frame rate for real- time detection of targets. For example if a IR imaging sensor operating at 60 H_Z,it can provide one image frame at every 16 ms. In this case the CR stage should process an image frame within 16 ms to identify the pixels that might contain targets.

CR stage identifies the pixels with targets by comparing pixel information with the estimated background information. The large deviation of pixel information from the background information probably due to the presence of a target. There exists several techniques in literature for the detection of point targets from IR image sequences. These techniques can be broadly categorized into:

• Spatial filtering techniques.
• Temporal filtering techniques.

2.1 Spatial Filtering Techniques

The spatial filtering techniques uses the spatial information, in neighbourhood surrounded by pixel under test (PUT) for estimating the background information.

The background information from spatial neighbourhood piexls can be estimated by simple mean or median filtering^3,4. Figure 2, depicts a 5x5 spatial neighbourhood surrounded by a PUT ‘S1’. The mean, median filtering schemes are easier to implement but will result very less SNR due to the background estimation inaccuracies. These inaccuracies are due to non-uniform and de-correlated backgrounds that are present surrounded by PUT. This is a common case in wide FOV sensors with less spatial resolution. However there exists other spatial filtering techniques, viz., image morphology based techniques⁵, Wavelet based decomposition techiques^6,7, they require significant information related to target. These techniques can also cause significant loss of target signature in the process of filtering. This can make detection of very low SNR targets difficult. If ATR systems require good detection range then spatial filtering schemes are not the obvious choice for target detection.

2.2 Temporal Filtering Techniques

Temporal filtering schemes^8,9 use information from a set of previous frames for estimating the background information at PUT. The temporal filtering schemes can provide the better estimate for the background due to high correlation between successively sampled images. Figure 3 depicts a temporal neighbourhood surrounded by a PUT.

For using temporal filtering scheme, the temporal images participating in background estimation process need to be co-registered with sub-pixel accuracy. The more the number of frames used for background estimation will yield better noise filtering. But, too many previous frames can make the background estimation inaccurate due to increase in registration errors.

There exists various other techniques, viz., hypothesis testing^10,11, track before detect^12-14, techniques that use both the spatial and temporal information^15,16 surrounded by PUT for accurately estimating the background information. Hypothesis testing methods are probabilistic techniques that use predefined models for the background to identify target pixels. These techniques have limited applications due to difficulty in modelling the IR clutter using a particular type of distribution. The airborne ATR system has to detect the targets against various backgrounds, like sky, vegetation, sea, industrial, rural, etc. Modelling these wide variety of backgrounds under wide variety of atmospheric conditions is highly impractical. Track before detect (TBD) techniques, on the other hand, require some a priori information related to target (like the target velocity in image plane). The target motion in image plane depends on the aspect angle, and it is highly variable. The TBD techniques uses multiple filter banks for various target velocities in the process of tracking. The tracking in image plane may fail in case of large manoeuvrability of platform carrying the ATR system. These limitations make TBD techniques difficult to implement.

In the present work authors have proposed a method for real-time target detection based on the change detection. The underlying principle behind the proposed approach is that a moving target can introduce certain change from one image to the other image due to its motion or variation in its signature. Figure 4 depicts the proposed clutter-rejection framework.

The proposed approach involves a background-estimation process and thresholding the residual image for identifying the possible target pixels.

2.3 Background Estimation

In our approach, the background is estimated by the temporal method. The temporal method uses the moving property of the target, and similarity of the background for two consecutive frames. The main challenge involved in the change-detection method is to align two consecutive frames over time for observing the change. In the present work the two consecutive frames were aligned using a registration process proposed¹⁸.

The selected image registration approach requires the platform motion to align the previous frame to the current frame. The platform motion data can be obtained using an onboard inertial navigation system. The registered previous image can give the better estimate for background due to stationary nature of background within one frame acquisition time. Following the registration process, the aligned images are subjected to subtraction for eliminating the background. The residual image is further subjected to threshold operation.

2.4 Threshold

The residual background image will be subjected to threshold operation to identify the pixels with significantly high residual values. The pixels with significantly high signal-to-noise ratio (SNR) values in residual image can be possibly due to the presence of the target. The noise statistics were estimated using neighbourhood pixels surrounded by PUT. Figure 5 depicts the local neighbourhood used for estimating the noise statistics.

The pixels values having significant SNR metric are declared as detections in a frame.

$\begin{array}{l}SNR=\frac{{\displaystyle \sum _{i=1}^9{S}_i}}{{\text{σ}}_{noise}}\hfill \\ {\text{σ}}_{noise}=std(N_1\mathrm{......}{N}_{16})\hfill \end{array}$

The tracking and association phase operate on the detections provided by the clutter-rejection module. The goal of tracking and association module¹⁷ is to associate detections into the existing tracks and to estimate, maintain (deletion, creation) the track states. The track state estimation can be performed using various prediction filters, viz., fixed gain α-β-γ filter, variable-gain Kalman filter, and particle filters, etc. The tracking and association phase also handles missing detections due to occlusions or target intensity variations for tracking with minimum error. The output from tracking and association phase is database of tracks. A track in the tracks database contains all the information related to the target.

The framework for the tracking module is depicted in Fig. 6. The tracking of the detections need to be performed independent of platform motion. In our approach, inertial coordinate system used is local level local north (LLLN). The clutter-rejection phase provides the detection locations as image coordinates (row, column). In the detection-transformation block, the detection locations are transformed from image coordinate (row, column) to inertial space (azimuth, elevation). The tracking and association module handles the detections efficiently by tracking continuously the targets even in the presence of platform manoeuvres, missing detections, handling sensor noise and track separation or loss conditions.

3.1 Track Initiation Phase

The initiation phase is executed only once when detection set is received for the first time and initialises the track structure, viz., gate width, gate centre, prediction filter. The gate is a region around the predicted detection position where the detection is expected to appear in the next frame. The tracking space is expressed in inertial coordinate (azimuth, elevation) system.

3.2 Track Association

The track association phase associates the detections to their respective gates. The gate is defined as a cone in the inertial space. The detection is associated to a track provided the angle between the gate centre and detection is less than the gate size. The association algorithm implemented is nearest- neighbour approach. The various association rules considered in our approach are

• Associating single detection in a single gate,
• Single detection in multiple gates,
• Multiple detection within a single gate, and
• Multiple detections within multiple gates.

3.3 Track Update Phase

The track update phase estimates target state variables (position, velocity, acceleration) using Kalman Filter¹⁷. The target-tracking process is defined through various track states as track undefined, track init, tracking mode and monitoring. The track is initially in undefined state, upon reception of first detection enters to init mode and a gate is initialised. The tracking mode starts after receiving the next detection, wherein the position, velocity, and acceleration are predicted, corrected, and estimated by Kalman filter. The track enters into monitoring state if it stops receiving a valid detection. The track not receiving a valid detection consequently for a defined period is switched to an undefined state.

The large track loss condition occurs due to non-availability of the initial target state (angular rate) information of the target. This causes initial tracking error to be large before the filter reaches steady state. In our proposed approach the initial estimate of the target state (angular rates) is estimated using the first two detections of the track. Initializing the filter with rough estimate of target angular rates can significantly reduce the filter convergence time.

Performance evaluation of algorithms is a challenging task, as it is difficult to obtain data sets representing all the target occurrence scenarios practically. In the present work, to evaluate the algorithms for all possible target occurrence scenarios, a synthetic data set generation module has been developed. This module uses the scene-rendering technique to implant the targets into recorded background videos. Figure 7 demonstrates block diagram of target implantation module. The target implanted video recordings are used for the algorithm evaluation.

The proposed target detection approach is evaluated on background recordings with simulated targets. Figures 8 and 9 demonstrate the effectiveness of the proposed detection process and background estimation approach. Figure 8(a) presents the current image; Fig. 8(b) presents the registered previous image into current image. Figure 9, presents the residual current image after background suppression. From change detection map presented in Fig. 9, it is obvious that residual image after background suppression contains minimal amount of clutter information (almost zero). This approach effectively reduces the clutter and improves the target SNR significantly.

The proposed target detection approach based on the temporal information has been compared with various other spatial filtering techniques using receiver operating characteristics (ROC). For any given detector, the trade-off between probability of detection (P_d) and probability of false alarm (P_f) can be described using ROC curves. The ROC curve plots, P_d(η) and P_f(η) as function of all possible values of threshold (η). Terms commonly used in ROC analysis are given by

True positives (TP) = Number of correct detections.

True negatives (TN) = Number of correct rejections.

False alarm (FA) = Number of false detections.

False negatives (FN) = Number of missed detections.

Probability of detection (P_d) = TP/(TP+FN)

Probability of false alarm (P_f) = FA/(FA+TN)

From the ROC curves presented in Fig. 10, it is observed that proposed temporal filtering approach is exhibiting significantly high P_d than other spatial filtering techniques for the same amount of false alarm. This improvement is mainly due to improved SNR achieved using the proposed temporal background subtraction approach.

The tracking performance was tested on synthetic data, generated by implanting targets into collected background recordings. The track loss rate was significantly reduced using proposed work compared to classical Kalman filter (KF) with nearest neighbour association (NNA). The performance achieved with the proposed tracking algorithm is presented in Table 1. Targets are implanted into collected background video recordings at various ranges from platform.

From the Table 1 it is concluded that proposed approach can track the targets present at various ranges without track loss. Whereas the conventional KF+NNA is having significant track loss, the significant improvement in tracking performance with the proposed approach is mainly due to the initialization of filter states with rough estimate of target angular rates.

The detection tracks generated by tracking association module will be classified into true target track or false target track by extracting various features related to target. As the paper’s main focus is on detection and tracking of targets from images, feature extraction and discrimination is not discussed by authors in detail. The feature extraction and discrimination is very subjective to application and target of interest.

The paper explains the importance of ATR systems based on the IR imaging. Analgorithmic framework for real time target detection and tracking is presented in the paper. The various algorithmic stages and their functions are presented. The algorithm framework is modeled using MATLAB tool. The data set for evaluating the algorithm components were generated by implanting targets in the background recording. The results obtained at various stages of the algorithm have been also discussed.

1. Brown, W. & Swonger, C. A prospectus for automatic target recognition. IEEE Trans. Aero. Electro. Sys., 1989, 25,401-410. doi: 10.1109/7.30795

2. Bhanu, B. Automatic target recognition: State-of-the-art survey. IEEE Trans. Aero. Electro. Sys., AES-22, 1986, (4), 364-379. doi: 10.1109/taes.1986.310772

3. Suyog, D.D.; Meng, H.E.; Ronda, V. & Philip, C. Max- mean and Max-median filters for detection of small targets. In Proceeding SPIE 3809, Signal and Data Processing of Small Targets. 1999, 74. doi: 10.1117/12.364049

4. Soni, T.; Zeidler, J.R. & Ku, W.H. Adaptive whitening filters for small target detection. In Proceeding SPIE 1698, Signal and Data Processing of Small Targets, 21, 1992. doi: 10.1117/12.139383

5. Victor, T.; Tamar, P. ; May, L. & Bondary, K. Morphology- based algorithm for point target detection in infrared backgrounds. In Proceeding of SPIE, Signal and Data Processing of Small Targets, 1993, 1954, pp. 2-11. doi: 10.1117/12.157758

6. Wang, Y. & Zhang, J. Dim target detection system based on DSP. In Proceeding Second symposium on International Computer Science and Computational Technology, Academy Publisher, 2009.

7. Marco, S. Del & Agaian, S. The design of wavelets for image enhancement and target detection. In Proceeding SPIE, Mobile Multimedia/Image Processing, Security, and Applications. 2009, 7351, pp.12-26.

8. Blostein, S.A. Detecting small moving objects in image sequences using sequential hypothesis testing. IEEE Trans. Signal Process., 1991, 39, 1611-1629. doi: 10.1109/78.134399

9. Ralph, J.F.; Smith M.I. & Heather, J.P. Motion-based detection, identification, and tracking for missile warning system applications. In Proceeding SPIE 5809, Signal Processing, Sensor Fusion, and Target Recognition XIV, 2005, 54. doi: 10.1117/12.603320

10. Tzannes, A.P. & Mooney, J.M. Point target detection in IR image sequences using spatio-temporal hypothesis testing. Air Force Research Laboratory, Hanscom AFB, MA, 1999.

11. Silverman, J.; Caefer, C.E.; DiSalvo, S. & Vickers, V.E. Temporal filtering for point target detection in staring IR imagery: II. Recursive variance filter. In Proceeding SPIE 3373, Signal and Data Processing of Small Targets 1998, 44. doi: 10.1117/12.324648

12. Kligys, S.; Rozovsky, B. & Tartakovsky, A. Detection algorithm and track before detect architecture based on non linear filtering for infrared search and track systems. Centre for applied mathematical sciences, University of southern california, Los Angeles, CA, 1998.

13. Warren, R.C. A bayesian track before detect algorithm for IR point target detection. Weapons Systems Division, Aeronautical and Maritime Research Laboratory, Victoria, Austalia, 2002. doi: 10.1007/978-3-7908-1782-9_39

14. Geo, S. & Shui, P.L. Method for moving point target detection in image sequences based on directional cumulation. In Proceeding SPIE 6795, 2I-1-6795 2I-6, 2007.

15. Tartakovsky, A.G. & Blazek, R.B. Effective adaptive spatial-temporal technique for clutter-rejection in IRST. In Proceeding SPIE 4048, Signal and Data Processing of Small Targets 2000, 85. doi: 10.1117/12.392023

16. Aridgides, T.; Fernandez, M.F.; Randolph, D. & Bray, D. Adaptive three-dimensional spatio-temporal filtering techniques for infrared clutter suppression. In Proceeding SPIE 1305, Signal and Data Processing of Small Targets 1990, 63. doi: 10.1117/12.21580

17. Brookner, E. Tracking and Kalman filtering made easy. John Wiley Sons Inc, New York, 1998. doi: 10.1002/0471224197

18. Shankar, Ravi & Raji, R. Registration of airborne IR images using platform attitude information. Def. Sci. J., 2014, 64(2),130-135. doi: 10.14429/dsj.64.5460

Authors thank Ms J. Manjula Outstanding Scientist and Director, Defence Avionics Research Establishment and Mr P.M. Soundar Rajan, DRDO Fellow, for their constant guidance and encouragement for carrying out the work reported here.