Skip to content

Commit

Permalink
Initial changes
Browse files Browse the repository at this point in the history
  • Loading branch information
JosePizarro3 committed Mar 27, 2024
1 parent 4980f35 commit 8782e3a
Show file tree
Hide file tree
Showing 16 changed files with 1,331 additions and 186 deletions.
26 changes: 9 additions & 17 deletions .github/workflows/actions.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,6 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
coveralls --service=github
- name: Test with nomad
run: |
python -m nomad.cli parse tests/data/test.example-format.txt
build-and-install:
runs-on: ubuntu-latest
steps:
Expand All @@ -42,23 +39,18 @@ jobs:
- name: Install the package
run: |
pip install dist/*.tar.gz --index-url https://gitlab.mpcdf.mpg.de/api/v4/projects/2187/packages/pypi/simple
- name: Test with nomad
run: |
python -m nomad.cli parse tests/data/test.example-format.txt
ruff-formatting:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: chartboost/ruff-action@v1
with:
args: "format ."
ruff-linting:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: chartboost/ruff-action@v1
with:
args: "check ."
version: 0.1.8
# to enable auto-formatting check, uncomment the following lines below
# ruff-formatting:
# runs-on: ubuntu-latest
# steps:
# - uses: actions/checkout@v3
# - uses: chartboost/ruff-action@v1
# with:
# args: "format . --check"
# version: 0.1.8
args: "format . --check"
version: 0.2.0
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -128,3 +128,6 @@ dmypy.json

# Pyre type checker
.pyre/

# VSCode launch json
.vscode/launch.json
24 changes: 24 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
{
"python.defaultInterpreterPath": ".pyenv/bin/python",
"python.terminal.activateEnvInCurrentTerminal": true,
"editor.rulers": [90],
"editor.renderWhitespace": "all",
"editor.tabSize": 4,
"[javascript]": {
"editor.tabSize": 2
},
"files.trimTrailingWhitespace": true,
"files.watcherExclude": {
"${workspaceFolder}/.pyenv/**": true
},
"files.exclude": {
"\"**/*.pyc\": {\"when\": \"$(basename).py\"}": true,
"**/__pycache__": true,
"**/node_modules": true
},
"python.testing.pytestPath": "pytest",
"python.testing.pytestArgs": ["tests"],
"python.testing.unittestEnabled": false,
"editor.defaultFormatter": "charliermarsh.ruff",
"editor.formatOnSave": true
}
121 changes: 65 additions & 56 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,88 +1,103 @@
# NOMAD's parser example plugin
![](https://github.com/nomad-coe/nomad-parser-magres/actions/workflows/actions.yml/badge.svg)
![](https://coveralls.io/repos/github/nomad-coe/nomad-parser-magres/badge.svg?branch=main)
[![NOMAD](https://img.shields.io/badge/Open%20NOMAD-lightgray?logo=data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4KPCEtLSBHZW5lcmF0b3I6IEFkb2JlIElsbHVzdHJhdG9yIDI3LjUuMCwgU1ZHIEV4cG9ydCBQbHVnLUluIC4gU1ZHIFZlcnNpb246IDYuMDAgQnVpbGQgMCkgIC0tPgo8c3ZnIHZlcnNpb249IjEuMSIgaWQ9IkxheWVyXzEiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIgeG1sbnM6eGxpbms9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkveGxpbmsiIHg9IjBweCIgeT0iMHB4IgoJIHZpZXdCb3g9IjAgMCAxNTAwIDE1MDAiIHN0eWxlPSJlbmFibGUtYmFja2dyb3VuZDpuZXcgMCAwIDE1MDAgMTUwMDsiIHhtbDpzcGFjZT0icHJlc2VydmUiPgo8c3R5bGUgdHlwZT0idGV4dC9jc3MiPgoJLnN0MHtmaWxsOiMxOTJFODY7c3Ryb2tlOiMxOTJFODY7c3Ryb2tlLXdpZHRoOjE0MS4zMjI3O3N0cm9rZS1taXRlcmxpbWl0OjEwO30KCS5zdDF7ZmlsbDojMkE0Q0RGO3N0cm9rZTojMkE0Q0RGO3N0cm9rZS13aWR0aDoxNDEuMzIyNztzdHJva2UtbWl0ZXJsaW1pdDoxMDt9Cjwvc3R5bGU+CjxwYXRoIGNsYXNzPSJzdDAiIGQ9Ik0xMTM2LjQsNjM2LjVjMTUwLjgsMCwyNzMuMS0xMjEuOSwyNzMuMS0yNzIuMlMxMjg3LjIsOTIuMSwxMTM2LjQsOTIuMWMtMTUwLjgsMC0yNzMuMSwxMjEuOS0yNzMuMSwyNzIuMgoJUzk4NS42LDYzNi41LDExMzYuNCw2MzYuNXoiLz4KPHBhdGggY2xhc3M9InN0MSIgZD0iTTEzMjksOTQ2Yy0xMDYuNC0xMDYtMjc4LjgtMTA2LTM4Ni4xLDBjLTk5LjYsOTkuMy0yNTguNywxMDYtMzY1LjEsMTguMWMtNi43LTcuNi0xMy40LTE2LjItMjEuMS0yMy45CgljLTEwNi40LTEwNi0xMDYuNC0yNzgsMC0zODQuOWMxMDYuNC0xMDYsMTA2LjQtMjc4LDAtMzg0LjlzLTI3OC44LTEwNi0zODYuMSwwYy0xMDcuMywxMDYtMTA2LjQsMjc4LDAsMzg0LjkKCWMxMDYuNCwxMDYsMTA2LjQsMjc4LDAsMzg0LjljLTYzLjIsNjMtODkuMSwxNTAtNzYuNywyMzIuMWM3LjcsNTcuMywzMy41LDExMy43LDc3LjYsMTU3LjZjMTA2LjQsMTA2LDI3OC44LDEwNiwzODYuMSwwCgljMTA2LjQtMTA2LDI3OC44LTEwNiwzODYuMSwwYzEwNi40LDEwNiwyNzguOCwxMDYsMzg2LjEsMEMxNDM1LjQsMTIyNCwxNDM1LjQsMTA1MiwxMzI5LDk0NnoiLz4KPC9zdmc+Cg==)](https://nomad-lab.eu/prod/v1/staging/gui/)

## Getting started

### Fork the project

Go to the github project page https://github.com/nomad-coe/nomad-parser-plugin-example, hit
fork (and leave a star, thanks!). Maybe you want to rename the project while forking!

### Clone your fork
# NOMAD's parser for magres

Follow the github instructions. The URL and directory depends on your user name or organization and the
project name you choose. But, it should look somewhat like this:
This is a parser plugin for the [magres file format](https://www.ccpnc.ac.uk/output/magres) developed in the CCP-NC for Nuclear Magnetic Resonance (NMR) data.

```
git clone git@github.com:markus1978/my-nomad-schema.git
cd my-nomad-schema
```
## Getting started

### Install the dependencies

You should create a virtual environment. You will need the `nomad-lab` package (and `pytest`).
You need at least Python 3.9.
Clone the project and, in the workspace folder, create a virtual environment (note this project uses Python 3.9):
```sh
git clone https://github.com/nomad-coe/nomad-parser-magres.git
cd nomad-parser-magres
python3.9 -m venv .pyenv
```

Install the `nomad-lab` package:
```sh
python3 -m venv .pyenv
source .pyenv/bin/activate
pip install --upgrade pip
pip install '.[dev]' --index-url https://gitlab.mpcdf.mpg.de/api/v4/projects/2187/packages/pypi/simple
```

**Note!**
Until we have an official pypi NOMAD release with the plugins functionality. Make
sure to include NOMAD's internal package registry (e.g. via `--index-url`). Follow the instructions
in `requirements.txt`.
> **Note**: Until we have an official PyPI NOMAD release with the plugin functionality, make
sure to include NOMAD's internal package registry (via `--index-url` in the above command).


### Run the tests

Make sure the current directory is in your path:
You can run the unit testing using the `pytest` package:

```sh
export PYTHONPATH=.
python -m pytest -sv
```

You can run automated tests with `pytest`:

We recommend to install the `coverage` and `coveralls` packages for a more comprehensive output of the testing:
```sh
pytest -svx tests
pip install coverage coveralls
python -m coverage run -m pytest -sv
```

### Run linting
## Development

The plugin is still under development. If you would like to contribute, install the package in editable mode (with the added `-e` flag) with the development dependencies:

```sh
ruff check .
pip install -e .[dev] --index-url https://gitlab.mpcdf.mpg.de/api/v4/projects/2187/packages/pypi/simple
```

### Run auto-formatting

This is entirely optional. To add this as a check in github actions pipeline, uncomment the `ruff-formatting` step in `./github/workflows/actions.yaml`.
### Setting up the plugin on your local installation
Read the [NOMAD plugin documentation](https://nomad-lab.eu/prod/v1/staging/docs/howto/oasis/plugins_install.html) for all details on how to deploy the plugin on your local NOMAD installation.

```sh
ruff format .
```
To deploy the plugin in your local NOMAD installation, follow the next steps:

You can parse an example archive that uses the schema with `nomad`
(installed via `nomad-lab` Python package):
1. Add the configurations related to NOMAD. This is already implemented through the plugin definition file ```src/nomad_parser_magres/nomad_plugin.yaml```:

```sh
nomad parse tests/data/test.example-format.txt --show-archive
```
```yaml
plugin_type: parser
name: parsers/magres
description: |
This plugin is used to parsed magres files into the NOMAD schema.
```
and the ```nomad.yaml``` configuration file:

## Developing your schema
```yaml
plugins:
include:
- 'parsers/magres'
options:
parsers/magres:
python_package: nomad_parser_magres
```
3. Add to your local NOMAD installation the same lines of your plugin ```nomad.yaml``` file.
4. Add to your local NOMAD installation environment the `PYTHONPATH` to your plugin. This can be done either by running the following command every time you start a new terminal for running the appworker, or by adding it to your virtual environment in the `<path-to-local-nomad-installation>/.pyenv/bin/activate` file:

You can now start to develop you schema. Here are a few things that you might want to change:
```sh
export PYTHONPATH="$PYTHONPATH:<path-to-nomad-parser-magres-cloned-repo>/src"
```

- The metadata in `nomad_plugin.yaml`.
- The name of the Python package `nomadparserexample`. If you want to define multiple plugins, you can nest packages.
- The name of the example parser `ExampleParser` and the schema definitions it might define.
- When you change module and class names, make sure to update the `nomad_plugin.yaml` accordingly.
If you are working in this repository, you just need to activate the environment to start working using the ```nomad-parser.magres``` package locally in your own Python scripts.

### Run linting and auto-formatting
Ruff auto-formatting is also a part of the GitHub workflow actions. Make sure that before you make a Pull Request to add your contributions to this repo, the following commands run in your local without any errors otherwise the workflow action will fail.
```sh
ruff check .
```
```sh
ruff format . --check
```

Alternatively, if you are using VSCode as your IDE, we added the settings configuration file, `.vscode/settings.json`, such that it performs `ruff format` whenever you save progress in a file.


<!--
## Build the python package

The `pyproject.toml` file contains everything that is necessary to turn the project
into a pip installable python package. Run the python build tool to create a package distribution:
The `pyproject.toml` file contains everything that is necessary to turn the project into a pip installable python package. Run the python build tool to create a package distribution:

```
pip install build
Expand All @@ -95,11 +110,5 @@ You can install the package with pip:
pip install dist/nomad-schema-plugin-example-1.0.tar.gz
```

Read more about python packages, `pyproject.toml`, and how to upload packages to PyPI
on the [PyPI documentation](https://packaging.python.org/en/latest/tutorials/packaging-projects/).


## Next steps

To learn more about plugins, how to add them to an Oasis, how to publish them, read our
documentation on plugins: https://nomad-lab.eu/docs/plugins/plugins.html.
Read more about python packages, `pyproject.toml`, and how to upload packages to PyPI on the [PyPI documentation](https://packaging.python.org/en/latest/tutorials/packaging-projects/).
-->
6 changes: 3 additions & 3 deletions nomad.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ plugins:
# We only include our schema here. Without the explicit include, all plugins will be
# loaded. Many build in plugins require more dependencies. Install nomad-lab[parsing]
# to make all default plugins work.
include: 'parsers/example'
include: 'parsers/magres'
options:
parsers/example:
python_package: nomadparserexample
parsers/magres:
python_package: nomad_parser_magres
8 changes: 0 additions & 8 deletions nomadparserexample/nomad_plugin.yaml

This file was deleted.

21 changes: 0 additions & 21 deletions nomadparserexample/parser.py

This file was deleted.

Loading

0 comments on commit 8782e3a

Please sign in to comment.