Defence Science Journal, Vol. 64, No. 5, September 2014, pp. 451-457, DOI : 10.14429/dsj.64.6873
© 2014, DESIDOC
Received 07 April 2014, Revised 04 July 2014, Online published 19 September 2014
Frequency and Spatial Domains Adaptive-based Enhancement Technique for Thermal Infrared Images
Low contrast and noisy image limits the amount of information conveyed to the user. With the proliferation of digital imagery and computer interface between man-and-machine, it is now viable to consider digital enhancement in the image before presenting it to the user, thus increasing the information throughput. With better contrast, target detection and discrimination can be improved. The paper presents a sequence of filtering operations in frequency and spatial domains to improve the quality of the thermal infrared (IR) images. Basically, two filters - homomorphic filter followed by adaptive Gaussian filter are applied to improve the quality of the thermal IR images. We have systematically evaluated the algorithm on a variety of images and carefully compared it with the techniques presented in the literature. We performed an evaluation of three filter banks such as homomorphic, Gaussian 5×5 and the proposed method, and we have seen that the proposed method yields optimal PSNR for all the thermal images. The results demonstrate that the proposed algorithm is efficient for enhancement of thermal IR images.
Thermal infrared radiance is emitted from a scene as a function of temperature and scene composition. Radiation from an ideal blackbody is exponentially proportional to temperature, which is typically controlled by topography in a terrestrial scene. Although the shape of the blackbody spectrum changes with temperature, the proportion of total radiance at a given wavelength changes negligibly over the range of temperatures common in a typical terrestrial scene.
Process of image enhancement does not increase the amount of information in the image. The image enhancement process modifies certain features thus making it easier to interpret visually and some details become discernible. Though the theoretical background for thermal image enhancement is the same as digital image processing theory but the algorithms that work well for visual images cannot be directly adapted to thermal image processing. It is very important to carefully select the algorithms/methods and modify these taking into account specific properties of thermal images1-3. Enhancement of thermal IR images of emitted radiation (typically at wavelengths from 4 μm to 14 μm) presents special difficulties because there is less spectral variation from pixel-to-pixel, compared to visible/near-infrared images of reflected sunlight (0.4 μm – 2.4 μm).
Various image enhancement techniques are reported by Tyagi and Amhia9. Dulski3, et al. proposed a four-steps algorithm for feature extraction from a thermal image. It starts with noise removal, contrast enhancement, edge detection and sharpening, and optimisation of gray scale palette at the end. The most important stage in the algorithm is contrast enhancement because of low-contrast thermal images. Most effective and most frequently used method of contrast enhancement is histogram equalisation5. In low-contrast images, the features of interest may occupy only a relatively narrow range of gray scale, with the majority of gray levels occupied by ‘uninteresting areas’ such as background and noise. These ‘uninteresting areas’ may also generate large counts of pixels, and hence, large peaks in the histogram. In this case, the global histogram equalisation amplifies the image noise and increases visual graininess or patchiness. The global histogram equalization technique does not adapt to local contrast requirements, and minor contrast differences can be entirely missed when the number of pixels falling in a particular gray range is small. This method is effective when applied to visual images, but unfortunately when it comes to thermal images, it improves rather background contrast, not the contrast of an object.
Adaptive histogram equalisation is a modified histogram equalisation procedure that optimises contrast enhancement based on local image data. The basic idea behind the scheme is to divide the image into a grid of rectangular contextual regions, and to apply standard histogram equalisation in each. The optimal number of contextual regions and the size of the regions depend on the type of input image, and the most commonly used region size is 8×8 (pixels). In addition, a bi-linear interpolation scheme is used to avoid discontinuity issues at the region boundaries.
Zuiderveld4 proposed a modified version of histogram equalised method in an adaptive way. This method is better than global histogram equalization method. But when the moving window position is on the border of object and background then the quality of the enhanced image is degraded and it becomes blurred. The contrast enhancement is based on plateau histogram and is described by Bing-Jian6, et al. It has all the advantages of typical histogram equalisation but does not excessively strengthen the contrast. This is achieved by applying plateau threshold, which is determined automatically on the basis of image histogram properties.
In recent years, infrared imaging has become an important tool for surveillance and monitoring, process industrial applications, defence applications, etc21. Jadin and Taib7 proposed a method for enhancing the warm regions in IR image of electric equipment while at the same time reducing the effect of unwanted background. Li8, et al. proposed a gradient-based approach for feature enhancement in a thermal image. They used the statistical properties of gradient of foreground object profiles and formulated object features with gradient saliency. A nonlinear image enhancement technique for enhancement IR image has been proposed by Erkanli10, et al. They proposed the nonlinear log transform that converts an original image into an adjusted image by applying the log function to each pixel of the original image. Agaian and Roopaei11 proposed an enhancement technique of thermal IR images using optimised stretching, filtering, and colour transformation techniques.
It is observed that the sequence of image enhancing operations is also important. If the sequence of operations is not correctly defined, the results may be below expectation and assumed level of image enhancement will not be achieved. The present paper aims to improve the image quality by contrast enhancement and noise filtering. The authors systematically evaluated the algorithm on a variety of images and carefully compared it with the techniques presented in the literature. The authors performed an evaluation of three filter banks, such as homomorphic, Gaussian 5×5 and the proposed method, and it was found that the proposed method yields optimal PSNR for all the thermal images. The results demonstrate that the proposed algorithm is efficient for enhancement of thermal IR images.
Thermal IR imaging sensor works in a completely passive manner by utilising the thermal contrast of a target against its background and is independent of prevailing light conditions. It generates extensive ranges for detection and recognition of targets in day and night conditions. In addition, long-wave IR (LWIR) (8-12 μm / 3-5 μm) can penetrate mist and fog considerably better than visible/near-IR light. Artificial smoke screening of targets is relatively ineffective during observation through thermal imagers. But thermal imaging technology is much more complex compared to image intensifiers in terms of IR detector and cryogenics, opto-mechanics, and advanced signal processing requirements.
First-, second-, and third-generation thermal imagers are used to describe different developmental stages of thermal imagers which are primarily based on IR detector configuration/geometry and means of scanning and read-out electronics techniques. The thermal images were recorded using the FLIR second-generation thermal imager in ground range. The specification of the FLIR imager are spectral band 7.7 μm - 10.5 μm, FLIR FOV 8°×6°(N), 16°×12°(W), detector 288×4 MCT (linear mercury cadmium telluride) LFPA, range 4 km (detection). The camera settings were: the emissivity = 0.92°, reflected temperature = 15 °C, and relative humidity = 50 per cent. Thermal sensitivity of the thermal camera at 50 Hz was 0.08 °C at 30 °C. The image recording rate was 50 frames per second. The thermal imager has given the desired recognition ranges of 4 km against tank targets. Qualities of the original images are not good enough and so image enhancement technique is required to improve the image quality.
Image enhancement is the process by which one tries to improve an image so that it looks subjectively better. One does not really know how the image should appear, but one can tell that whether it has been improved or not, by considering, for example, whether more detail can be seen, or whether unwanted flickering has been removed, or the contrast is better etc. The approach largely depends on what one wishes to achieve. In general, there are two major approaches: those which reason about the statistics of the gray values of the image, and those which reason about spatial frequency content of the image. The proposed enhancement technique to enhance the thermal IR image is a sequence of filter operations - homomorphic filter followed by adaptive Gaussian filter. The proposed technique is described in the following sub-sections.
2.1 Homomorphic Filter
Images are sometimes been acquired under poor illumination. Under this condition, the same uniform region will appears brighter on some areas and darker on others. This undesired situation will lead to several severe problems in computer vision based system. The pixels might be misclassified, leading to wrong segmentation results, and therefore contribute to inaccurate evaluation or analysis from the system. Therefore, it is very crucial to process these type of images like thermal IR images before doing segmentation etc. The basic method for image enhancement is based on gray level intensities and contrast transformations, edge emphasising, noise reduction, etc. Some improvement in this domain is realised introducing an algorithm for automatic global contrast enhancement12.
For medical images obtained by x-rays, good results are achieved using the modification of homomorphic filtering functions in the logarithmic domain13. The combination of two nonlinear filter types – wavelet transformation and homomorphic filtering also-give improved results14. Using homomorphic filtering, ant-isotropic filtering, and algorithms based on the wavelet transformation, it is possible to improve the weak illumination and contrast of images15. Peli and Lim16 have proposed an approach to the image enhancement using homomorphic filtering and local characteristics modifications, performed by nonlinear coefficients multiplication. Micic17, et al. proposed an algorithm for automatic determination of nonlinear homomorphic filter coefficients used for local contrast enhancement in digital image processing.
Most effective method of thermal IR image enhancement is histogram equalisation. But the background of a scene is improved more than the object in the scene of a thermal IR image by this method. No methodology is available in the literature for enhancing thermal IR image using homomorphic filter, and to improve the quality of thermal IR image, the authors have used homomorphic filtering technique in this paper.
One knows that an image function can be represented using illumination and reflectance components. The illuminations of an image generally represents the slow varying details of the image, while the reflected component represents sharp details and are abruptly varying junctions of dissimilar objects. Hence one can interpret that the illumination corresponds to low frequency components and reflectance corresponds to high frequency components in the frequency domain, respectively. It is possible to develop a filter which will control both high-and low-frequency components, which is homomorphic filter. The equation for homomorphic filter can be derived from the illumination reflectance model given by the equation18
where i(x, y)and r(x, y) are the illumination and reflectance components, respectively. Homomorphic filtering technique aims to reduce the significance of i(x, y) by reducing the low frequency components of the image. This can be achieved by executing the filtering process in frequency domain. To process an image in frequency domain, the image needs first to be transformed from spatial domain to frequency domain. This can be done using Fourier transform. However, before the transformation is taking place, logarithm function has been used to change the multiplication operation of r(x, y) with i(x, y) in Eqn. (1) into addition operation.
In general, homomorphic filtering can be implemented using five steps, as follows20:
Step 1: Take a natural logarithm of both sides of Eqn. (1)
Step 1: Use the Fourier transform to transform the image into frequency domain.
where Fi(u, v) and Fr(u, v) are the Fourier transforms of ln i(x, y) and ln r(x, y), respectively.
Step 3: High pass the Z(u, v) by means of a filter function H(u, v) in frequency domain, and get a filtered version S(u, v) as the following.
Step 4: Take an inverse Fourier transform to get the filtered image in the spatial domain.
Step 5: The filtered enhanced image g(x, y) can be obtained using the following equations.
In this particular application, the key to the approach is that separation of the illumination and reflectance components, is achieved in the form in Eqn. (4).
However, in literature, there are several variations of equations that have been used to present H(u, v) in Eqn. (5). In this paper, the typical filter for homomorphic filtering process has been used18. This filter has circularly symmetric curve shape, centered at (u, v) = (0,0) coordinates in frequency domain. This filter is modified from Gaussian high pass filter, which is known as Difference of Gaussian (DoG) filter. The transfer function for DoG filter is defined as
where constant c has been introduced to control the steepness of the slope, D0 is the cut-off frequency, D(u, v) is the distance between coordinates (u, v) and the centre of frequency at (0,0). For this filter, three important parameters are needed to be set by the user. These are the high frequency gain γH , the low frequency gain γL, and the cut-off frequency D0. If γH is set greater than 1, and γL is set lower than 1, the filter function tends to decrease the contribution made by the illumination (which occupies mostly the low frequency components) and amplify the contribution made by the reflectance (which occupies most of the high frequency components). At the end, the net result will be a simultaneous dynamic range compression and contrast enhancement. The value of the low frequency gain should be set such as γL = 0.45, to halve the spectral energy of the illumination, and the value of high frequency gain is set such as γH = 2.6 to double the spectral energy of the reflectance components18. Fan and Zhang19 suggested that the value of c is 0.5 and in this paper also we has taken c = 0.7. In practice, all these three parameter values are often determined empirically and there is no clear way to choose the exact suitable values for these parameters.
The objectives to introduced homomorphic filter model in the proposed algorithm have two aspects.
- Images that are useful to be enhanced using homomorphic filtering, are images where the illumination is distributed unequally causing the objects in the image to appear in a dark colour. In general, these images are dark, and so their details are hidden.
- Homomorphic filtering in a frequency-domain filtering process that compresses the brightness from the lighting condition while enhancing the contrast from the reflectance properties of the object.
2.2 Adaptive Gaussian Filter
It is a fact that the images are often corrupted by random variations in intensity values, called noise. Some common types of noise are salt and pepper noise, impulse noise, and Gaussian noise. Salt and pepper noise contains random occurrences of both black and white intensity values. However, impulse noise contains only random occurrences of white intensity values. Unlike these, Gaussian noise contains variations in intensity that is drawn from a Gaussian or normal distribution, and is a very good model for many kinds of sensor noise, such as the noise due to camera electronics. Thermal images are effected by this type of noise. So for removing such type of noises, Gaussian filter is very much essential and effective in thermal IR image.
Gaussian filters are a class of linear smoothing filters with the weights chosen according to the shape of a Gaussian function. The Gaussian smoothing filter is a very good filter for removing noise drawn from a normal distribution. The twodimensional discrete Gaussian function is defined as18
where is the distance of the neighbourhood pixel (x, y) from the centre pixel (xc, yc) of the output image where the filter is being applied.
To create a mask for filtering, the Gaussian is defined centered on the origin as
The Gaussian filter is really interesting filter because weights are not homogeneous and gives more influence to the closer pixels and less to the others. The advantages of Gaussian filtering are : Gaussian smoothing is very effective for removing Gaussian noise. Filtering weights decrease monotonically from central peak, giving most weight to central pixels. The weights give higher significance to pixels near the edge (reduces edge blurring). It is a linear low-pass filter. Rotationally symmetric (perform the same in all directions). Computationally efficient since Gaussian is separable. The degree of smoothing is controlled by standard deviation a and window size.
The window size and standard deviation σ are two parameters of Gaussian filter technique by which the quality of the image is changed. If we considered very small window size, then we can see that this will not influencing a lot the image. This filter with very small σ is not smoothing the image because the weights for the neighboring pixels are really small. On the other hand, if σ gets bigger, so does the window size, the smoothing effect is more visible and impact. But at the same time the filtered image will be blurred. So the choice of window size and standard deviation σ are two most important parameters for Gaussian smoothing filter.
Normally, first a global standard deviation σ is calculated from the image. Then within a certain window of fixed size and using a fixed σ with Gaussian kernel, the weights are calculated for all pixels of the window. Typically, the same pattern of weights is used in each window, which means that the linear filter is spatially invariant and can be implemented using a convolution mask.
The proposed technique is adaptive-based Gaussian filter technique in which in stead of global σ, we are calculating local σ within a moving window. That is, first we have taken a moving window of size k×k (k is odd integer). Then at the current position of the window, we have calculated σ and the weights are computed according to a Gaussian function (Eqn 10). Convolved by the weighted mask of size k×k at the current position (centre pixel of the window) of the image and replaced the current position image intensity value by the calculated weighted average value of the window. Next, move the window to the next pixel and repeat the same operation unless all pixels of the original image are considered. The procedure of the adaptive Gaussian filtering technique is shown in Fig. 1.
The advantages by the proposed adaptive Gaussian filter are as follows:
- The standard deviation σ is very sensitive parameter in Gaussian filter and the choice of s is a critical issue. In the proposed technique, σ is not a user-defined parameter and it is calculated from the local gray value distribution of the original image.
- Within a local homogeneous region, the value of σ is small and the region is less affected by the noise. So in such a case, less smoothing effect of the region by the Gaussian filter, which is not possible by the global σ-based Gaussian filter.
- When the local region is heterogeneous then the value of σ will be more. In such case, the smoothing effect is more visible and impact of the region by Gaussian filter, which may not be always possible by the global σ-based Gaussian filter.
Inherently, the human visual system is limited to the visible region. However, technological developments in the last century and thereafter has enabled mankind to extend its imaging capability to wavelengths on either side of the visible spectrum, both towards the shorter (UV and X-Ray region) and to the longer, i.e. IR radiations, mm wave and microwave region. Thermal imagings are concerned with the range of wavelengths emitted by objects by virtue of their temperatures and emissivity. So the thermal image is a pictorial representation of the temperature and emissivity differences between the object and its surroundings. The problem with the thermal IR images is that these are not enough good quality for further processing, and hence, image enhancement is very much essential step.
To test the efficiency of the proposed technique, several data are considered with different sizes. Figure 2(a) shows an original human and building thermal IR image of size 292×237. The result of the adaptive histogram equalisation method4 is shown in Fig. 2(b). Figure 2(c) shows the output of the original image by applying Gaussian filter with window size 5×5 and σ = 1.5. Whereas, the output of the proposed adaptive-based Gaussian filter is shown in Fig. 2(d), which is visibly better than Fig. 2(c). Figure 2(e) shows the output of the original image by applying homomorphic filter. Finally, Fig. 2(f) shows the output of the original image by applying the proposed enhancement technique. Though analytical measurement regarding the quality of the enhancement process is very difficult unless onee knows the ideal image but one can realised the quality improvement through visual impact. It has been seen that enhancement quality by the proposed method significantly improves the visual appearance of the image than other techniques as shown above.
Figure 2. Human and building thermal IR image: (a) original image, (b) adaptive histogram equalisation image, (c) Gaussian filter image by 5x5 window with o = 1.5, (d) proposed adaptive Gaussian method without homomorphic filter, (e) homomorphic filter image without adaptive Gaussian method, and (f) proposed method.
Other sets of thermal IR images are shown in Figs 3(a), 4(a), and 5(a) (tower, garden, and tower thermal IR images, respectively) and the corresponding processed images by the proposed technique are shown in Figs 3(b), 4(b), and 5(b), respectively. All the scenes are taken by a long-range thermal imager (LRTI) at a distance 16 km. By close observations, it has been seen that the tower in Fig. 3(a) is not clearly visible and at the same time, the tower and the steps of the tower have been clearly visible in the processed image Fig. 3(b). Similarly, for the other two images (Figs. 4(b) and 5(b)) the objects like barricades of small bamboo pools in Fig. 4(b), the tower in Fig. 5(b), which is totally non-visible in Fig. 5(a) and the backgrounds of all the images are not clearly visible in their corresponding original images. But the same are cleared and details have been enhanced by the proposed method.
An example of an image frame from a real video sequence captured at a distance 4.7 km through turbulent atmosphere in sunny day time (12 Noon) is shown in Fig. 6(a) (moving tank thermal image). The same frame generated by the proposed technique is shown in Fig. 6(b). The image contains noise due to turbulence and the moving tank (BMP 2) is not much visible due to noise. The turbulence removal result is shown in Fig. 6(b) where most of the turbulence has been removed and also the tank is clearer from the smooth background than the original image frame.
An evaluation of three filter banks such as Homomorphic, Gaussian 5×5 and the proposed method were performed and Table 1 shows the PSNRs obtained using three filter banks such as Homomorphic, Gaussian 5×5, and the proposed method for each thermal image. The proposed filter bank yields optimal PSNR for all the thermal images.
The detection, acquisition, classification, aim point selection of ground mobile, high-value targets in high clutter, and adverse weather environment, are critical issues in the development of smart weapons. Advanced image processing algorithms will be a major element for the success of the smart weapons. We have presented an approach for enhancing the thermal IR image. The proposed method is a sequence of filtering operations to improve the quality of the thermal IR image. Mainly, two filters are applied in the proposed technique – homomorphic filter followed by adaptive Gaussian filter. Our algorithm was applied on various thermal IR images of different distances, weather, and timings conditions. We have also compared the results with other existing methods of enhancement and also we have calculated the PSNRs using three filter banks such as homomorphic, Gaussian 5×5 and the proposed method for each thermal image. The proposed filter bank yields optimal PSNR for all the thermal images. The results demonstrate that the algorithm is better than the other and also the objects are clearer than the original images, which are useful for further processing.
Our future work includes the objects from the results we have obtained in this paper and the development of classifiers to identify the different objects. Improving the detection accuracy of the classifier scheme, multi-hypotheses, which contain tanks, electric towers, construction area in a deep forest, human, and video tracking, and the more useful features of training images for data modelling, will be the main focus of our future study.
1. Accetta, J.S. & Shumaker, D.L. The infrared and electrooptical systems handbook. Ann Arbor MI and SPIE Press, Bellingham, WA, 1993.
2. Holst, G.C. Testing and evaluating of infrared imaging systems. SPIE Optical Engineering Press, 1998.
3. Dulski, R.; Powalisz, P.; Kastek, M. & Trazaskawka, P. Enhancing image quality produced by IR cameras. In Proceedings of SPIE, 2010, 7834. [Full text via CrossRef]
4. Zuiderveld, K. Contrast limited adaptive histogram equalization. Graphics Gems IV, Academic Press, 1994, 474-485.
5. Weith-Glushkod, S. & Salvaggio, C. Quantitative analysis of infrared contrast enhancement. In Proceedings of SPIE, 2007, 6543. [Full text via CrossRef]
6. Bing-Jian, W.; Shang-qian, L.; Qing, L. & Hui-xin, Z. A real-time contrast enhancement algorithm for infrared images based on plateau histogram. Infrared Phys. Technol., 2006, 48(1), 77-82. [Full text via CrossRef]
7. Jadin, M. S. & Taib, S. Infrared image enhancement and segmentation for extracting the thermal anomalies in electrical equipment. Electron. Electr. Eng., 2012, 120(4), 107-112. [Full text via CrossRef]
8. Li, Z.; Zhang, J.; Wu, Q. & Geers, G. Feature enhancement using gradient salience on thermal image. IEEE Digital Image Comp.: Techn. Appl., 2010, 556-562. [Full text via CrossRef]
9. Tyagi, S. & Amhia, H. Image enhancement and analysis of thermal images using various techniques of image processing. Int. J. Eng. Res. Appl., 2013, 3(2), 579-584.
10. Erkanli, S.; Li. J. & Oguslu, E. Fusion of visual and thermal images using genetic algorithms. In Bio-Inspired Computational Algorithms and Their Applications, edited by S. Gao, 2012. pp. 187-212. [Full text via CrossRef]
11. Agaian, S. & Roopaei, M. Novel infrared and thermal image enhancement algorithms. In Proceedings of SPIE, Mobile Multimedia/Image Processing, Security, and Applications, 2013, 8755. [Full text via CrossRef]
12. Grundland, M. & Dodgson, N.A. Automatic contrast enhancement by histogram warping. Comp. Vision Grap., 2006, 293-300. [Full text via CrossRef]
13. Surobhi, N.A. & Amin, Md. R. Employment of modified homomorphic filters in medical imaging. Daffodil Int. Uni. J. Sci. Technol., 2006, 1(1), 60-63.
14. Yoon, J. H. & Ro, Y. M. Enhancement of the contrast in mammographic images using the homomorphic filter method. IEICE Trans. Inf., 2002, E-85D(1).
15. Padmavathi, G.; Subashini, P.; Kumar, M.M. & Thakur, S.K. Comparison of filters used for underwater image pre-processing. Int. J. Comput. Sci Network Secur., 2010, 10(1), 58-65.
16. Peli, T. & Lim, J.S. Adaptive filtering for image enhancement. J. Opto. Eng., 1982, 21(1), 108-112. [Full text via CrossRef]
17. Micie, A.D.; Dordevic, B.R.; Lekic, P.N. & Andelkovic, B.R. Automatic determination of filter coefficients for local contrast enhancement. Trans. Famena XXXVII-1, 2013, 63-76.
18. Gonzalez, R. C. & Woods, R. E. Digital image processing. Prentice Hall, 2008.
19. Fan, C.N. & Zhang, F.Y. Homomorphic filtering based illumination normalization method for face recognition. Pat. Recog. Lett., 2011, 32(10), 1468-1479. [Full text via CrossRef]
20. Saleh, S. A. M. & Ibrahim, H. Mathematical equations for homomorphic filtering in frequency domain: A literature Survey. In the International Conference on Information and Knowledge Management (ICIKM) 2012, IPCSIT, 45, IACSIT Press, Singapore, pp. 74-77.
21. Dulski, R.; Sosnowski, T.; Pialkowski, T. & Milewski, S. Evaluation of hardware implementation of the infrared image enhancement algorithm. QIRT 2012, ID - 215.