Add NNCG to optimizers submodule

[pratikrathore8]

No description provided.

[lululxvi]

Format the code using black https://github.com/psf/black

[pratikrathore8]

I've formatted the code with black in the latest commit!

[lululxvi]

If this only supports pytorch, then it should be in https://github.com/lululxvi/deepxde/tree/master/deepxde/optimizers/pytorch

[pratikrathore8]

Moved to the right folder!

[pratikrathore8]

Made the formatting changes. I noticed there are other lines in the code with > 88 chars per line. Would you like for me to fix those?

[lululxvi]

Made the formatting changes. I noticed there are other lines in the code with > 88 chars per line. Would you like for me to fix those?

Yes.

Could you also fix the Codacy issues?

[pratikrathore8]

The formatting has been fixed.

There are still some Codacy issues, but these are mainly due to "too many local variables" and the __init__ function in the optimizer, which is just following PyTorch conventions.

What do we need do next to further integrate NNCG into the codebase?

[lululxvi]

Need to add the NNCG to https://github.com/lululxvi/deepxde/blob/master/deepxde/optimizers/pytorch/optimizers.py

[pratikrathore8]

I've added NNCG to optimizers.py and NNCG_options to config.py

[pratikrathore8]

Fixed!

[pratikrathore8]

Fixed!

[lululxvi]

Have you tried using NNCG in some demo examples?

[pratikrathore8]

A quick update:

I've started thinking about how to make a demo example that uses NNCG. The plan would be to modify some of the existing examples (e.g. Burgers.py) and add on NNCG after Adam and L-BFGS.

I'll have to make some modifications to model.py to full integrate NNCG into the training pipeline. I'm aiming to have an example fully working by the end of next week!

[pratikrathore8]

@lululxvi Made the changes. Should we consider merging _train_pytorch_nncg into _train_sgd by adding an argument to _train_sgd that decides whether or not to perform a backward pass on the loss? Or would you prefer we leave the functions separate, despite the code duplication?

[pratikrathore8]

@lululxvi I added a train_step_nncg function as you suggested. NNCG now runs using _train_sgd.

Thanks for your patience with the refactoring steps so far!

[pratikrathore8]

@lululxvi Done!

[lululxvi]

Add a document for the example similar to this https://deepxde.readthedocs.io/en/latest/demos/pinn_forward/burgers.html

[pratikrathore8]

@lululxvi I added the demo. It's basically the same as the one you linked, except I added the part with NNCG to the end.

Is there a way to check if it's been properly integrated into the docs?

[lululxvi]

Please resolve the conflict.

You can look at the generated doc by clicking "Details" at the right side of "docs/readthedocs.org:deepxde" in in the check list.

[pratikrathore8]

@lululxvi I've fixed the merge conflict!

I took a look at the doc for the demo, and it seems good to me.

[pratikrathore8]

These are all great suggestions! I've incorporated them into my most recent commit.

[pratikrathore8]

Good catch! I made the fix in my most recent commit.