Parameter estimation of ordinary differential equations in NeuroMANCER

Differentiable models such as Neural ordinary differential equations (NODEs) or neural state space models (NSSMs) represent a class of black box models that can incorporate prior physical knowledge into their architectures and loss functions. Examples include structural assumption on the computational graph inspired by domain application, or structure of the weight matrices of NSSM models, or networked NODE architecture. Differentiaity of NODEs and NSSMs allows us to leverage gradient-based optimization algorithms for learning the unknown parameters of these structured digital twin models from observational data of the real system.
In this tutorial, we show how to estimate parameters of differentiable equation models from time series data.
code examples: github.com/pnnl/neuromancer/t...
This work was partially supported by the Mathematics for Artificial Reasoning in Science (MARS) and Data Model Convergence (DMC) initiatives via the Laboratory Directed Research and Development (LDRD) investments at Pacific Northwest National Laboratory (PNNL), by the U.S. Department of Energy, through the Office of Advanced Scientific Computing Research's “Data-Driven Decision Control for Complex Systems (DnC2S)” project, and through the Energy Efficiency and Renewable Energy, Building Technologies Office under the “Dynamic decarbonization through autonomous physics-centric deep learning and optimization of building operations” and the “Advancing Market-Ready Building Energy Management by Cost-Effective Differentiable Predictive Control” projects.
PNNL is a multi-program national laboratory operated for the U.S. Department of Energy (DOE) by Battelle Memorial Institute under Contract No. DE-AC05-76RL0-1830.

[@salmankhalildurrani]

great video

[@neuromancer_SciML]

Thank you! I am glad you found it useful.

[@edwardmorgan426]

I have a complicated grey physics box for my dynamics which is not pytorch supported. Actually, the function is generated from casadi. For parameters which exist within this box, how can i use this box with the 'requires_grad nn parameters'??

[@neuromancer_SciML]

For parameter estimation of ODEs and gray-box universal differential equations (UDEs)
see our examples in:
github.com/pnnl/neuromancer/tree/master/examples/ODEs
For a more detailed response please open a new discussion under Q&As and provide more details about your use case
github.com/pnnl/neuromancer/discussions

[@abdullahcerkezoglu]

Thank you for the content. Can you by any chance do another example of Nonautonomous system identification where the system is not fully observable ?

[@neuromancer_SciML]

Hi Abdullah, thank you for the inquiry. We will prepare this example for the upcoming release.
As of now, we have one example that allows you to use different types of learnable embeddings to estimate the latent state of the dynamical system.
github.com/pnnl/neuromancer/blob/master/examples/system_identification/two_tank_neural_ssm.py
you can check line #78 for the estimator options and line #166 for the instantiation of these estimators, which are implemented here:
github.com/pnnl/neuromancer/blob/bc45a2772009303da6aac129617d82dec08e7d7e/neuromancer/estimators.py

[@user-xk1bp8yx1w]

I plan to do black box system identification. Does the MLP work in place of duffing parameter in this example?

[@neuromancer_SciML]

Hi Pallavi. Certainly so.
We have two examples of black-box system ID using MLPs in neural ODEs and neural state space models
github.com/pnnl/neuromancer/blob/master/examples/system_identification/two_tank_node.py
github.com/pnnl/neuromancer/blob/master/examples/system_identification/two_tank_neural_ssm.py
hope that helps :)

[@pallavighimire6809]

@@neuromancer_SciML I tried running the first example and I am stuck at Ambiguous Look up error .

[@pallavighimire6809]

This error is obtained when I try to obtain yhat.

[@Urenduil]

@@pallavighimire6809 can you please create an issue on GitHub with a complete error message and description of your environment setup? Thank you.

[@neuromancer_SciML]

@@pallavighimire6809 Can you please create an issue on github with full error message and description of you setup?

[@sumitkumar-ct7ch]

Can we use the Neuromancer software with jupyter notebook ?

[@neuromancer_SciML]

Absolutely yes. See our current google colab examples github.com/pnnl/neuromancer/tree/master#features-and-examples

[@viniciusviena8496]

It is a great piece of software you built. I would like to try it, but I am a fervorous Julia user in sciml. What are the advantage of this package over Julia
sciml ecosystem (Diffeqflux, optimization.jl, Differential equations.jl, SciMLsensitivity.jl)?

[@neuromancer_SciML]

Thank you, Vinicius.
Our framework is to a great extent inspired by Julia's SciML ecosystem. I am a big fan of Julia :)
What we aim to create with NeuroMANCER is an open-source ecosystem for SciML in PyTorch that will allow for easy integration of user-defined AI models with constrained optimization for dynamical systems and parametric optimal control. What is unique about NeuroMANCER is that it aims to integrate all of these domains in a single SW framework.
We do so by providing high-level symbolic mathematical language used in the constrained optimization frameworks on top of Pytorch modules and callables.
We are currently conducting active research in the "learning to optimize with constraints" and "learning for model-based optimal control" and aim to continuously expand our library in this direction.
Compared to what we provide, Julia packages are light years ahead in the differential equation capabilities, numerical solvers, and overall performance.
Our framework is a young research project, but we hope to attract the interest of the Python community to facilitate open-source development and integration of the latest SciML methods that are being developed in PyTorch.

[@viniciusviena8496]

@@neuromancer_SciML I really appreciate your prompt and rich answer. these are great features and plans for the framework . I will play with it a bit to get familiar and maybe I can reach a level that allows me to contribute to it. Julia sciml ecosystem has been built over many years from many developers, so it's just a matter of time for Neuromancer to get as many feature as Julia. Really appreciate the things you are doing. Congratulations!!

[@neuromancer_SciML]

@@viniciusviena8496 Thank you so much! We would be most happy to attract the interest of the open-source community. All feedback and contributions are most welcome! There is much to develop and improve :)