Geometric Clifford Algebra Networks and Clifford Neural Layers for PDE Modeling

Join the Learning on Graphs and Geometry Reading Group: hannes-stark.com/logag-readin...
Paper: "Geometric Clifford Algebra Networks"
arxiv.org/abs/2302.06594
Abstract: We propose Geometric Clifford Algebra Networks (GCANs) that are based on symmetry group transformations using geometric (Clifford) algebras. GCANs are particularly well-suited for representing and manipulating geometric transformations, often found in dynamical systems. We first review the quintessence of modern (plane-based) geometric algebra, which builds on isometries encoded as elements of the Pin(p,q,r) group. We then propose the concept of group action layers, which linearly combine object transformations using pre-specified group actions. Together with a new activation and normalization scheme, these layers serve as adjustable geometric templates that can be refined via gradient descent. Theoretical advantages are strongly reflected in the modeling of three-dimensional rigid body transformations as well as large-scale fluid dynamics simulations, showing significantly improved performance over traditional methods.
Authors: David Ruhe, Jayesh K. Gupta, Steven de Keninck, Max Welling, Johannes Brandstetter
Paper: "Clifford Neural Layers for PDE Modeling"
arxiv.org/abs/2209.04934
Abstract: Partial differential equations (PDEs) see widespread use in sciences and engineering to describe simulation of physical processes as scalar and vector fields interacting and coevolving over time. Due to the computationally expensive nature of their standard solution methods, neural PDE surrogates have become an active research topic to accelerate these simulations. However, current methods do not explicitly take into account the relationship between different fields and their internal components, which are often correlated. Viewing the time evolution of such correlated fields through the lens of multivector fields allows us to overcome these limitations. Multivector fields consist of scalar, vector, as well as higher-order components, such as bivectors and trivectors. Their algebraic properties, such as multiplication, addition and other arithmetic operations can be described by Clifford algebras. To our knowledge, this paper presents the first usage of such multivector representations together with Clifford convolutions and Clifford Fourier transforms in the context of deep learning. The resulting Clifford neural layers are universally applicable and will find direct use in the areas of fluid dynamics, weather forecasting, and the modeling of physical systems in general. We empirically evaluate the benefit of Clifford neural layers by replacing convolution and Fourier operations in common neural PDE surrogates by their Clifford counterparts on two-dimensional Navier-Stokes and weather modeling tasks, as well as three-dimensional Maxwell equations. Clifford neural layers consistently improve generalization capabilities of the tested neural PDE surrogates.
Authors: Johannes Brandstetter, Rianne van den Berg, Max Welling, Jayesh K. Gupta
Speakers: Johannes Brandstetter - / jo_brandstetter
Twitter Hannes: / hannesstaerk
Twitter Dominique: / dom_beaini
Twitter Valence Discovery: / valence_ai
Reading Group Slack: join.slack.com/t/logag/shared...
~
Chapters
00:00 - Intro
03:35 - Why Clifford Algebras for Deep Learning
12:26 - Introduction to Clifford Algebras
21:23 - Clifford Convolution
24:44 - Clifford Fourier Transform & Results
39:52 - Follow Up Work
41:29 - Geometric Clifford Algebra Networks
44:16 - The Pin(n) Group and Transformations
54:39 - The Overall Picture
57:34 - Group Action Layers
01:10:16 - Fluid Dynamics
01:14:16 - Q+A

[@Mr-iw2kg]

Thanks, very nice presentation.
Min 27:58, here I think for the Stokes eq. we clearly have an interaction between the pressure field (scalar) and the velocity field (vector).

[@philosophem1629]

Where i can find a series of video to start to understand what are you sayin?

[@romanavr]

there is rather popular video «swift introduction on geometric algebra» - it is a good place to start and get excited about the topic