Thinking Outside the Box Breaks Neural Network Barrier

Associate Professor of Mechanical and Aerospace Engineering Ankit Srivastava has developed a groundbreaking technique for solving equations describing systems that change with time, and NVIDIA has used the method to improve Modulus, their program for building artificial intelligence models.

Just about every system that changes with time—fluid flow, heat exchange, financial systems, and so much more—is described by a certain type of equation called a partial differential equation (PDE). 

Decades of research have established methods for solving PDEs that work but require significant time and computational resources. Despite that, they are the foundation of most engineering and physics simulations, including those used by large companies such as Ford and GM to simulate complex systems, such as cars and engines.

With the widespread adoption of neural networks across many computational fields, researchers have started to explore the possibility of using neural networks to solve PDEs. 

Until now, all attempts ran into a significant roadblock. 

“All they could show was that if you have a very simple domain, let’s say a square or a rectangle, then you can create neural networks which satisfies the boundary conditions on those domains. But real world problems are not simple, and they were never successful when it came to complex boundary conditions,” says Srivastava. “People wanted to do it, but they didn’t have the tools.”

Srivastava says his breakthrough in figuring out a solution came through conversations with N. Sukumar, a professor at University of California, Davis. While Sukumar does not work on neural networks, he has spent his career finding innovative ways to model complex geometries. 

Sukumar helped Srivastava figure out how to combine computer-aided design (CAD) software with a neural network, leading to a “very generalizable” solution, allowing him to use a neural network to solve much more complex geometries than ever before. They are still testing the method, but in theory it should work for arbitrarily complex systems.

“They are very, very powerful,” says Srivastava. 

With its implementation into NVIDIA’s Modulus software, Srivastava says the method has already been applied to problems in a range of areas including fluid mechanics, structural dynamics, and finance. 

It will take some work to get to the level of simulating a whole engine, including figuring out how to solve multi-phase problems with a neural network, but Srivastava says he thinks it’s only a matter of years. Current methods for solving PDEs spend millions of dollars and 80 percent of their computational resources on one step—creating the mesh—and neural networks don’t require a mesh, so figuring out how to apply them to these problems could mean major cost savings.

“I think that is the most important and exciting part, that this is truly the first time that we have a potential algorithm which can solve a problem without creating a mesh,” says Srivastava.

Image: (Left to right) The domain of a partial differential equation (PDE), an image created by DallE representing the neural network, the neural network’s solution to the PDE.