The paper presents advanced mission management system (MMS) for unmanned aerial vehicles, based on integrated modular avionics (IMA) architecture. IMA architecture enables the MMS to host high end functions for autonomous navigation and attack. MMS is a collection of systems to execute the mission objectives. The system constitutes mission computer (MC), sensors and other sub-systems. The MMS-MC needs to execute advanced algorithms like terrain referenced navigation, vision-aided navigation, automatic target recognition, sensor fusion, online path planning, and tactical planning for autonomy and safety. This demands high-end architecture in terms of hardware, software, and communication. The MMS-MC is designed to exploit the benefits of IMA concepts such as open system architecture, hardware and software architecture catering for portability, technology transparency, scalability, system reconfigurability and fault tolerance. This paper investigates on advanced navigation methods for augmenting INS with terrain-referenced navigation and vision-aided navigation during GPS non-availability. This paper also includes approach to implement these methods and simulation results are provided accordingly, and also discusses in a limited way, the approach for implementing online path planning.

Keywords:    Mission management system,   integrated modular avionics,  mission computer,  terrain-referenced navigation  optical flow  online path planning

Mission management system (MMS) is a primary system to execute missions of the unmanned aerial vehicles (UAVs). The typical functions of MMS are navigation, flight guidance, conflict management, situation assessment and online path and tactical plannings. Figure 1 depicts the various functions of MMS. MMS gives the UAV necessary autonomy and intelligence by processing the information gathered through various onboard sensors. This demands complex hardware and software to implement ‘intelligent’ and ‘autonomous’ behaviour. The need for complexity management while keeping low costs requires new architectures and methods to cope with these requirements. Hence, MMS is designed to exploit integrated modular avionics (IMA) concept together with open system architectures. This paper presents the mission computer (MC) architecture and advanced navigation methods for MMS in the subsequent sections.

In current scenario the aircraft navigation is performed by the MC with the basic sensors like INS, GPS, RADALT, and air data system. The UAV is designed to navigate according to the planned flight plan. The guidance parameters are computed by the MMS using basic sensor parameters. Navigation with INS alone yields errors to the order of 1.8 Nm/hr in position^22,23. INS with GPS combination reduces the errors to a great extent with accuracy of around 15 m. The denial/non-availability of GPS is currently being considered as an important problem in navigation.

Terrain-referenced navigation can be considered as one of the important alternatives for GPS non-availability. Terrain- referenced navigation (TRN) is a function of digital terrain system (DTS) being developed for tactical aircraft operations which offers a range of capabilities in terms of both safety and tactical benefits. DTS is designed to increase the situational awareness, to improve navigation in GPS-denied environment, and to perform safe flying at low altitudes and in valleys. The paper further elaborates on usage of terrain for GPS outage conditions.

The UAV flight over the non-segregated air space and populated areas is a safety concern and has a direct bearing on the deployment policies. Vision systems are of great importance as these sort out problems like long-term GPS outages and replacement for pilot vision. The vision systems of today are capable of applications like visual odometry and mapping (vision-aided navigation), visual reference point updates, sense and avoid, and automated target recognition functions. This paper discusses the methods for implementing visual odometry. This technique uses image sequences from which the relative motion of the UAV is estimated, and hence, the relative displacements of the UAV are derived.

The success of a UAV mission is highly dependent on its ability to reach the target safely and to accomplish all mission objectives. In every mission, there is a possibility of unknown threats that may appear, due to which the UAV has to deviate from the original planned flight path. These threats are known as dynamic threats or pop-up threats, and these are namely, enemy ground radar, surface-to-air-missiles, enemy aircraft, terrain obstacles, etc. On detection of afore-mentioned threats, the system needs to assess the threat zone and implement the proper strategy to avoid the threat. The new route plan (online generated path) is a trade-off between safety and other constraints. The system has to consider various constraints such as flight dynamics, mission time, time-on-target (ToT), Fuel constraints along with terrain and various other safety constraints. This paper also discusses the algorithms to implement online path planning.

2.1 Hardware

The core hardware of the mission management system comprises mission computer (MC). The MC is designed considering basic design criteria as low-power dissipation, high-end computing core, high-speed switched fabric communication between internal modules, distributed data network for external communication, and high reliability system. The system, in particular its structure and system management functions, are based on the IMA concepts.

In this architecture¹, the computing capacity is concentrated in the processing module, which is common across all the modules. The mezzanine incorporates the basic input/output functions of the MC such as external communication and graphics. Each processing module being 6U card supports two mezzanines. The mezzanines are designed as per VITA 42.3¹⁷ and communicate with processing module through PCI Express interface. There are three types of mezzanines designed to meet the external communication requirements of MC - 1553 card, ARINC and DIO card. The structure of the MC is shown in Fig. 2, and discussed in the following text. The MC consists of four main types of hardware components:

• Processing module
• Mezzanine cards
• Backplane
• IO transition panel (IOTP)

Communications among processing modules within the MC takes place using the switch fabric - PCI Express and sRIO available with the processor. External communication options range from the conventional Mil-Std 1553¹⁸ command/ response bus or ARINC 429¹⁹ data distribution bus to avionics full duplex switched Ethernet, AFDX²⁰ on a high-capacity optical serial fibre channel network.

IMA systems offer fault tolerance² by reconfiguring on the occurrence of faults. MMS also implements fault tolerance, so that when a processing module becomes defective, the system reconfigures and a spare module takes over the functionality of the failed module and utilises the resources (mezzanines) of faulty module. Within the MC, a limited degree of interchangeability between components is offered using a number of identical VPX, SBCs, and XMC modules. Open standards used in the MC include commercial standards such as ATR, VITA, VPX, PCI Express²¹, XMC and AFDX²⁰, open MIL-Standards such as Mil-Std-1553B, and ARINC 429 standards.

2.2 Software

The software architecture is based on ARINC 653 compliant RTOS. The application process is the basic software element that is configurable in the system. The application process depends on the processor and communication resources and is executed when the management functions provide these resources. This approach provides deterministic scheduling, high integrity to application software modules, and also limits propagation of failures. This approach satisfies the conditions of high level of safety and security for a mission as well as for mixed critical software architecture.

Mission management system role is to execute mission as per an offline mission plan with the standard navigation sensors like INGPS, ADS, RADALT, laser ranger, and IFF. The advanced navigation functions discussed below provide a means of safety, decision-making and mission accomplishment.

3.1 Terrain-referenced Navigation

In our context terrain referenced navigation is a method for navigation in the GPS-denied environments where digital terrain elevation data (DTED) is used as a reference profile. The DTED is a standard data set of terrain elevation values in uniform matrix form, providing basic quantitative data for the applications that require terrain elevation, slope, and surface roughness information. Here, the measurements of the terrain height over mean sea-level are collected along the flight path of the aircraft and an estimate of the aircraft position is obtained by correlating these measurements with the digital terrain elevation data (DTED) values^3-5. The latitude and longitude of the current position of the aircraft can be calculated from the best correlation. Since the DTED measurements are considered as reference, the accuracy of the position estimate depends on DTED resolution. In accordance with the accuracy requirements different DTED levels are used. Our system has been modelled using DTED level-2 with a resolution of 30 m. The curvature-based approach^6,7 for terrain-referenced navigation is discussed below.

3.1.1 Curvature-based Approach

In our application, the terrain has been modelled using Gaussian and Mean curvature profiles. By this method, maximum and minimum curvature points can be extracted which are then used as feature points for correlating the sub-region with the reference DTED map. The curvature values are calculated by fitting a surface over the elevation data and calculating the first and second directional derivatives of this surface. This process needs to be carried out for both sub-region and on reference map (DTED) of region of interest.

To simulate the terrain matching as realistically as possible, the following scenario is assumed. First, the elevation data (DTED) of entire area of interest are obtained. Second, the elevation data for a small sub-region (online profile) is obtained by differencing the values of Barometric-altimeter and Radio-altimeter installed on the aircraft. Gaussian curvature (K) and Mean curvature (H) are used for feature extraction as these are invariant under translation and rotation. The steps involved are as follows.

Step 1: Surface Fitting

• Data smoothing is performed to remove random noise using averaging filter.
• Cubic Lagrange surface f(x, y) is fitted to the set of elevation points, as shown in Fig. 3.
• Directional derivatives are calculated on the interpolated points.

Step 2: Feature Extraction

• Gaussian curvature and mean curvature are calculated using differential geometry formula.

  Gaussian curvature (K) and Mean curvature (H) of a surface is given by:

  
  

  $K=\frac{f_{xx}{f}_{yy}\text{-}{f}_{xy}^2}{{(1+f_x^2+f_y^2)}^2}$

  
  

  and

  
  

  $H=\frac{(1+f_x^2)f_{yy}+(1+f_y^2)f_{xx}\text{-2}{f}_x{f}_y{f}_{xy}}{{(1+f_x^2+f_y^2)}^{3/2}}$



• The absolute value of K and H are obtained, i.e. |K| and |H|. f_xx, f_yy, f_xy being 2nd-order and f_x, f_y f being 1st- order derivatives at the point x, y on the surface f(x, y), respectively as shown in Fig. 4 and Fig. 5.
• Dynamic thresholding is performed as per the profile of the curvatures on the surface as shown in Fig. 6.
• The correlation between the patch and entire terrain is done with sufficient number of feature (curvature) points.

Step 3: Correlation Algorithm

The region of interest is constructed by determining the centroid and fitting a radius around it. The features of the region are total number of points above the threshold, their sign of curvature and the sum of distances from the centroid. These features are correlated with the reference DTED data.

After identifying the best match, the latitude, longitude, elevation, and error values are stored. These values are used to update the INS in the aircraft for accurate and precise navigation. The size of area of interest can be reduced based on the confidence level for more accurate and precise matching.

3.1.2 Simulation and Results

Lagrange surface fitted over 3x3 matrix, i.e., 9 terrain points, is shown in Fig. 3. The 2^nd-order directional derivative i.e. mean curvature ‘H’ after interpolation is shown in Fig. 4. Figure 5 depicts the Gaussian curvature ‘K’ after the interpolation. The signs and magnitude of Mean and Gaussian curvatures define all the surface-derived attributes. The Gaussian curvature defines more of peaks points, as shown in the Fig. 5. The Mean curvature shows more number of points, as it defines more abrupt changes as in Fig. 4. The mean curvature method uses higher threshold values to obtain precise match, whereas Gaussian curvature uses lower threshold values. After thresholding, correlation process yields match within 1 or 2 pixels.

3.2 Vision-aided Navigation - Visual Odometry

Visual odometry^10-12 refers to estimation of aircraft’s displacement using video stream obtained from an onboard camera. In this paper, an approach to estimate aircraft’s angular and linear velocities are presented from the optical flow. Optical flow¹³ refers to the movement of each pixel in subsequent image frames. Optical flow at each pixel depends on two things, first relative motion between aircraft and surroundings, and second, distance between aircraft and surroundings. A ‘laser range finder’ has been used to find this distance. With the distance is known, resulting equations are solved by least square method. Estimated velocities are integrated to obtain the displacements and position of the aircraft. Due to errors in measurement process, estimated velocities have inherent errors. Error in position accumulates over time due to integration of erroneous velocities. This is overcome by the process of mapping.

In mapping, a stored database of satellite images. Images are appropriately scaled and adjusted for aircraft’s attitudes. Image obtained from the onboard camera is correlated with the stored satellite images. Output of mapping process is used to update the result of visual odometry by a Kalman filter.

3.2.1 Optical Flow Method

Following steps illustrate the complete process.

Step 1 : Calculation of Optical Flow: Block matching algorithm is proposed for obtaining the optical flow. Image is divided in small rectangular blocks. In the subsequent image frames these blocks would have shifted by different amounts. It is this shifting that constitutes the optical flow. This optical flow is measured by correlating the rectangular blocks in consecutive concerned frames.

Step 1 : Correlation in the Optical Flow Estimation : Details of correlation algorithm as used in estimation of optical flow. The following expression computes the value of optical flow –

${\overline{V}}_m=\underset{\overline{v}}{\min }{\displaystyle \sum _{(x,y)\in B_m}{\left|L(\left(x,y\right)+\overline{v})-L(x,y)\right|}^2}$

1. B_m corresponds to rectangular blocks of image.
2. L(x,y) indicates the brightness value at pixel position (x,y).
3. $\overline{V}$ represents a 2-D vector consisting of movements on image plane in terms of pixels.
4. ${\overline{V}}_m$ is the amount (in terms of pixels) by which block B_m has shifted in next image frame, i.e., the estimated value of optical flow.

Following points are important in this correlation process.

• Search Region: A search region is predicted to carry out the correlation, based on aircraft present velocities.
• Frame Gap: It is the delay between the two image frames that are being correlated to calculate the optical flow.

If subsequent frames are being correlated continuously, the frame gap is one. If alternate frames are correlated, then frame gap is 2 and so on. Frame gap should be large enough, so that at least some optical flow is observed. At the same time, it should not be so large that the blocks jump-off the entire image.

1. Laser range finder is used to measure the distance between the aircraft and the ground.
2. Perspective projection model has been assumed for the camera. By projecting the ground points on camera image plane, the following equations were arrived, relating observed optical flow to aircraft velocities.

$\left[\begin{array}{cccccc}\left(\raisebox{1ex}{$d$}\!\left/ \!\raisebox{-1ex}{$Z$}\right.\right)& 0& \left(\raisebox{1ex}{$-x$}\!\left/ \!\raisebox{-1ex}{$Z$}\right.\right)& -\left(\raisebox{1ex}{$x^2+d^2$}\!\left/ \!\raisebox{-1ex}{$d$}\right.\right)& \left(\raisebox{1ex}{$xy$}\!\left/ \!\raisebox{-1ex}{$d$}\right.\right)& y\\ 0& \left(\raisebox{1ex}{$d$}\!\left/ \!\raisebox{-1ex}{$z$}\right.\right)& \left(\raisebox{1ex}{$-y$}\!\left/ \!\raisebox{-1ex}{$z$}\right.\right)& \raisebox{1ex}{$xy$}\!\left/ \!\raisebox{-1ex}{$d$}\right.& -\left(\raisebox{1ex}{$y^2+d^2$}\!\left/ \!\raisebox{-1ex}{$d$}\right.\right)& x\end{array}\right]*\left[\begin{array}{c}\dot{X}\\ \dot{Y}\\ \dot{Z}\\ \dot{H}\\ \dot{P}\\ \dot{R}\end{array}\right]=\left[\begin{array}{c}\dot{x}\\ \dot{y}\end{array}\right]$

where $s={\left[\dot{X}\dot{Y}\dot{Z}\dot{H}\dot{P}\dot{R}\right]}^{\text{T}}$ represents vector of six motion states of the aircraft. $[\dot{X}\dot{Y}\dot{Z}]$ represents three linear velocities and $[\dot{H}\dot{P}\dot{R}]$ represents three angular velocities along yaw, pitch, and roll axis. ‘x’ and ‘y’ represent the position on image in terms of pixels (x and y direction). $\dot{x}$ and $\dot{y}$ represent the optical flow along x and y direction at the point (x,y). 'd' is the distance between image plane and origin of camera axis. Z represents distance between aircraft and ground. d is a camera parameter and known. Z is measured by laser range finder. $\dot{x}$ and $\dot{y}$ is calculated in Step 1. S needs to be estimated.

1. For estimating S , above system is a set of two equations in six unknowns, and thus, an under-determined system. Performing measurements at more than three points gives more than six equations, and thus, make the system over determined. Due to inherent errors in measurements, this over-determined system is generally inconsistent. This inconsistency is resolved using least square solutions method. This also helps to get rid of errors.
2. The velocities obtained above are integrated to obtain the aircraft position and attitudes.

3.2.2 Simulation and Results

A synthetic video, generated on the principle of perspective projection, was used for testing of this algorithm. No aircraft model was used; instead, a pre-decided trajectory and attitudes were simulated. This will not affect the result as our method for visual odometry is not based on aircraft model. An image of ground was projected on the camera image plane at every point of trajectory. This sequence of projected images formed the required video for testing the above algorithm. Simulation results for roll estimation are shown in Fig. 7 and X and Y velocities estimation in Fig. 8.

3.3 Online Path Planning

Online path planning in the military framework is usually characterised by a substantial amount of risks (pop-up threats). These risks generate a mandatory requirement for online path planning with trade-off on some parameters like mission time and path length. The terrain below and dynamic constraints of the aircraft need to be considered as boundary conditions to plan a feasible path. Another important aspect integrated with iterative step method¹⁴ is an additional function to reduce the chance or avoid UAV being tracked by an enemy radar^15,16. A condition called “lock loss” is used by UAV to avoid being shot down by a missile. The approach used in this method is based on estimating the probability of the UAV being tracked by the radar over a period of time. For 3-D online path planning, the DTED map provides the vertical profile and the safety height to be considered along the planned path. The vertical profile data obtained from DTED map is integrated with iterative step method.

3.3.1 Iterative Step Method

In the iterative step method the actual state and the dynamic constraints of UAV are considered to define the possible paths over a safe area. Our application has been modelled for UAV’s dynamic constraint bounded by turn rates. The iterative step method is suggested as it is a better approach to handle the dynamic nature of the threat environment. The input data set for the method consists of threat type, position and velocity derived from various sensors like INGPS, ADS, RWR, DTED, vision system and RADALT. The threat posed by every obstacle around the UAV is assessed. The criteria to choose an optimum path is a tradeoff between safety and mission accomplishment on time. A risk probability map is estimated that defines the aircraft being vulnerable to threat as a function of the threat position (threat range). Integrating it with other parameters (flight path, vehicle dynamics, RCS etc.) the total risk probability is estimated. The risk map is computed that contains cost estimates of the parameters as function of threat position. The risk map is computed for every planned path and one with minimum risk is suggested as a safe path.

3.3.2 Results

For testing the above-mentioned method, it is assumed that the UAV is flying in a level flight. A radar threat is simulated along its path. For various bank angles and pitch angles, various paths available to the UAV are shown in Fig. 9. Each of the paths is rated with a probability of being tracked by the threat radar, and the path with the least probability of being tracked is chosen to achieve success in the mission. As the range of the UAV from the threat decreases, the probability of being tracked by the radar (threat) increases.

IMA-based mission computer as a functional prototype has been successfully realised in Defence Avionics Research Establishment, Banglore. The MC hardware is designed around common processing modules with provision for two functional modules (mezzanines). The functional redundancy has been increased through full mesh topology providing access to the mezzanine module by any of the processing module. Design and development effort for new functions has been reduced to a large extent due to this approach.

The work carried out to overcome GPS outages is perceived as an important technology in future avionics. The use of DTED maps for various avionics applications is a widespread concept. The curvature-based approach extracting the feature points with high value of Gaussian curvature K and Mean curvature H can determine the terrain to accuracy of 1 or 2 pixel. To handle the uncertainty in the curvature-based approach, Bayesian method²⁴ is suggested. The implementation of these methods requires very high processing power and parallelism.

The visual odometry method for estimating aircraft velocities based upon observed optical flow and integrating it with reference image mapping process can be a self-contained navigation method. The accuracy of this method basically depends on the optical sensors, image quality, ranger, optical flow estimation method, inertial measurement for image stabilisation and filters used for the estimation process. The fusion of parameter estimates from this process with INS parameters is in progress. We are working on image registration process to bound the errors from optical flow method.

The iterative method for online path planning gives alternate way points and trajectories to avoid the pop-up threats. The proposed method proves to be capable for producing feasible (collision-free) paths rendering it suitable for real-time calculations, and hence, particularly useful in online applications.

Authors thank Shri PM Soundar Rajan, Outstanding Scientist and Director, Defence Avionics Research Establishment, for his constant guidance and encouragement for carrying out the work reported here. Authors also thank all who have directly or indirectly contributed towards this manuscript.

1. López, Juan; Royo, Pablo; Barrado, Cristina & Pastor, Enric. Modular avionics for seamless reconfigurable UAS missions. In Proceeding of 27^th IEEE/AIAA Digital Avionics Systems Conference, 2008. [Full text via CrossRef]

2. Gangkofer, M.; Kader, H.; Klöckner W. & Whit, C.G. Transitioning to integrated modular avionics with a mission management system. In RTO SCI Symposium on Strategies to Mitigate Obsolescence in Defense Systems Using Commercial Components, 23-25 October 2000.

3. Priestley, N. Terrain referenced navigation. In Position Location and Navigation Symposium, IEEE PLANS ‘90, 20-23 March 1990, pp. 482-489. [Full text via CrossRef]

4. Hollowell, J. Heli/SITAN: a terrain referenced navigation algorithm for helicopters. In the Position Location and Navigation Symposium, Record. The 1990’s - A Decade of Excellence in the Navigation Sciences. IEEE PLANS ‘90. 20-23 March 1990. [Full text via CrossRef]

5. Pei, Yubo; Chen, Zhe & Hung, J. C. BITAN-II: An improved terrain aided navigation algorithm. In Proceeding of 22nd International Conference on Industrial Electronics, Control, and Instrumentation, IEEE IECON 1996. 5-10 Aug 1996. [Full text via CrossRef]

6. Goldgof, Dmitry B.; Huang, Thomas S. & Lee, Hua. A Curvature based approach to terrain recognition. IEEE Trans. Pat. Anal. Mach. Intel., 1989, 11(11), 1213-1217. [Full text via CrossRef]

7. Goldgof, D. B.; Huang, T. S. & Lee, H. Feature extraction and terrain matching. In Proceeding of Computer Society Conference on Computer Vision and Pattern Recognition, 1988. CVPR ‘88., Ann Arbor, MI, 5-9 Jun 1988. pp. 899 - 904. [Full text via CrossRef]

8. Yao, Y. S. & Chellapa, R. Dynamic Feature Point Tracking in an Image Sequence. In Proceeding of 12^th IAPR International Conference on Pattern Recognition, 1994, 1, pp. 654-657. [Full text via CrossRef]

9. Mendel, Jerry. Lessons in Estimation theory for signal processing communications and control. Pearson Education.

10. Scaramuzza, D. & Fraundorfer, F. Visual Odometry (Tutorial). IEEE Rob. Auto. Mag., 2011, 18(4), 80-92. [Full text via CrossRef]

11. Guizilini, V. & Ramos, F. Visual odometry learning for unmanned aerial vehicles. In IEEE International Conference on Robotics andAutomation (ICRA), 9-13 May 2011, pp. 6213-6220. [Full text via CrossRef]

12. Soatto, S.; Frezza, R. & Perona, P. Motion estimation via dynamic vision. IEEE Trans. Automatic Control, 1996, 41(3), 393-413. [Full text via CrossRef]

13. Horn, B. & Schunck, B. Determining optical flow. Artificial Intelligence, 1981, 17(1-3), 185-203. [Full text via CrossRef]

14. Weiß, B.; Naderhirn, M. & Luigi, del Re. Global real-time path planning for UAVs in uncertain environment. In the International symposium on Intelligent Control, 2006. [Full text via CrossRef]

15. Kabamba, P.T.; Meerkov, S.M. & Zeitz, F.H. Optimal path planning for unmanned combat aerial vehicles to defeat radar tracking. J. Guid. Cont., 2006, 29(2), 279-288. [Full text via CrossRef]

16. Moore, F.W. Radar cross section reduction Via route planning and intelligent control. IEEE Trans. Cont. Syst. Technol., 2002, 10(5), 696-700. [Full text via CrossRef]

17. ANSI/VITA 42.3-2006, XMC PCI Express Protocol Layer Standard.

18. MIL-STD-1553B: Digital time division command/ response multiplex data bus. United States Department of Defense, September 1978.

19. ARINC Specification 429, Parts 1-17. Annapolis, Maryland: Aeronautical Radio, Inc. 2004. pp. 2-5.

20. ARINC Specification 664P7-1. Aircraft Data Network, Part 7, Avionics Full-Duplex Switched Ethernet Network.

21. PCI Express specifications – PCIe Base Specification 3.0

22. Reddy, G. Satheesh & Saraswat, V.K. Advanced navigation system for aircraft applications. Def. Sci. J., 2013, 63(2), 131-137. [Full text via CrossRef] doi

23. SAGEM, Inertial Navigation System SIGMA 95N Technical Specification.

24. Bergman, Niclas; Ljung, Lennart & Gustafsson, Fredrik. Terrain navigation using Bayesian statistics. IEEE Control Syst., 1999, 19(3), 33-40. [Full text via CrossRef]