Imagine a remote pilot in the field, who has a time-sensitive mission, and whose main aerial platform (aircraft) has been compromised. The pilot has a collection of other aerial platforms that could potentially be used as alternatives, along with various sensor payloads, any one of which can be attached to a platform. A decision must be made rapidly: given the flight plan and other key mission parameters, which combination of aerial platform and sensor payload will allow the pilot to complete the mission at hand most effectively?
The standard way to answer this question is to run high-fidelity simulations for each of the platform and payload combinations, over the flight plan. However, this is very resource-intensive, requiring significant processing power, time, and often internet connectivity. Our pilot is in the field, has access to a standard laptop, and time is of the essence.
This theoretical challenge mirrors numerous, diverse real world scenarios in which an optimal design decision must be made. Broadly: out of nearly infinite possible options, what configuration of components and strategies will create the best possible helicopter / drone / submarine / spacecraft / microchip / bridge / power plant / solar array / etc. to accomplish XYZ? However, when time and/or resource constraints are present, optimality becomes much more difficult to achieve.
Over the past few years, as part of our work for Government clients, as well as Internal Research and Development (IR&D) efforts, Galois scientists have been exploring better, faster, cheaper ways to meet these sorts of design challenges through surrogate modeling, a promising approach, one variation of which we discussed in another blog article several months ago.
In our aerial platform example, a surrogate model can efficiently approximate the output of a high-fidelity simulator, with low computational overhead. As input, the surrogate model accepts a platform, payload, and flight plan, and as output, it provides a set of performance characteristics, such as mission success, time to completion, battery power remaining, and so forth. In short, the pilot can use a surrogate model to quickly find the optimal combination of aerial platform and sensor payload to effectively complete their mission.
Initial Steps
This promising automated problem-solving approach is generalizable to all sorts of design spaces: from aerospace to mission planning and beyond – all it takes is the right model and solid input data.
As the first step toward achieving our goal of formalizing a generalizable surrogate modeling approach, we have developed a proof-of-concept prototype, which combines an efficient, data-driven surrogate model with a quick-to-compute, low-fidelity physics-based simulator, and also includes visualization techniques from our collaborators at Charles River Analytics. In this prototype, rather than applying Koopman Theory (as discussed in the aforementioned article), we followed an approach based on transformer neural networks: akin to those used in large language models such as ChatGPT or GPT-4, but much smaller in size, and thus more efficient to run. As input, each neural network is given a detailed description of an aerial platform design, and as output, it computes some key performance characteristics for the platform: the maximum distance that the platform can fly, the maximum speed that it can achieve, how long it can hover, and so on. These predicted characteristics are then combined with payload information and flight plan details, and provided as input to a simplified physics-based simulator, which efficiently determines whether the mission will be successfully completed, and if so, how much battery power will remain upon completion.
As an example, consider a mission where an aerial platform must fly to a specific location, hover in the air to collect data, and then return to its starting point along a different route. The pilot has access to nine aerial platforms, and two sensor payloads; one of these payloads has a larger weight, but collects data faster, reducing the necessary hover time, whereas the second payload is lighter, but requires longer data collection times. In less than a minute of processing on standard hardware, our prototype produces estimates for the proportion of battery power that will remain at the end of the mission for each platform-payload combination.
The proportion of power remaining at the end of the simulated mission, for each combination of platform and payload.
In the figure above, each pair of bars corresponds to a platform, and each bar in the pair specifies a payload. The platforms are labeled with platform IDs. If a bar falls in the negative direction (power remaining < 0 on the y-axis), this indicates that the platform was able to collect data, but did not have enough battery power to return to its starting location; if a bar is absent, this means that there was not enough battery power to even complete data collection. We observe that the first platform is the most power efficient, with over half of the power remaining at the end of the mission. We also see that in most cases, it is more power efficient to use the heavier payload, because it reduces the required hover time. However, there is an exception: for platform 12095, the mission is successfully completed with the lighter payload, with just a little power left, whereas with the heavier payload, the platform runs out of power before completing the mission. We illustrate this latter scenario in the figure below; note that the data collection site is denoted by a large dot near the bottom right, while red is used to mark the portion of the path that was not flown, due to a depleted battery.
An example of an incomplete flight path, resulting from a depleted battery.
Future Vision
As one would expect with an initial, proof-of-concept prototype, there are currently a number of limitations. First, due to our relatively restricted scope of training data, some of the neural networks have limited prediction accuracy, and are unlikely to generalize outside of the dataset on which they were trained. The high-speed, low-fidelity simulator is also limited to two-dimensional simulations. This enables fast computation and comparison across platforms and payloads, but does not take into account all environmental factors that could affect system performance. Furthermore, this simulator relies on standard equations of flight for a fixed-wing aircraft, which would need to be generalized to fit unique aerial aircraft designs. Addressing these issues is a task for future research. In particular we envision a system where neural networks are trained on a much larger and broader set of data that includes not only platform design features, but also payload and flight path characteristics.
Furthermore, we have completed recent work (on another program) developing an approach to leverage physics-based models to design the architecture and verify dynamics of neural network models; we hypothesize that leveraging these techniques will both increase accuracy and reliability, while removing the need for a secondary simulator. While our initial use case has been on assisting remote pilots in effectively and rapidly making decisions for time-sensitive missions, we believe that this approach for surrogate modeling can provide value across a wide range of applications where we need to balance high speed with high fidelity, including rapid prototype evaluation for complex design and engineering workflows (from building aircraft to power plants to microchips), critical mission planning in dynamic environments, and even predicting disease and wildfire spread.
Stay tuned as we continue our research efforts!