Tk/experimental

[terrykong]

What does this PR do ?

Introduces experimental directories but verification related to the CI will come in a follow up PR. The idea being that experimental tests are allowed to fail

Before

NeMo-Aligner/
├── docs/
│   └── user-guide/
│       └── ppo.html
├── nemo_aligner/
│   ├── algorithms/
│   └── data/
│       └── datasets.py
└── tests/
    ├── test_datasets.py
    └── functional/
        └── test_cases/
            └── dpo-llama3

Proposal

NeMo-Aligner/
├── docs/
│   ├── user-guide/
│   │   └── ppo.html
│   └── user-guide-experimental/    <----- new doc dir
│       └── new-thing.html
├── nemo_aligner/
│   ├── algorithms/
│   ├── data/
│   │   ├── datasets.py
│   │   └── test_datasets.py        <----- tests colocated with module
│   └── experimental/               <----- mirrors the structure around it
│       ├── algorithms/
│       └── data/
│           ├── datasets.py         <----- mirrored datasets
│           └── test_datasets.py    <----- mirrored test file
└── tests/
    └── functional/
        └── test_cases/
            └── dpo-llama3

• By moving the docs out of the main dir, we can control what is rendered in the NeMo docs
• colocating the tests in the same package makes it easier to find tests and reduces number of mirrored dirs
  • these can be filtered out when we publish to pypi
• functional tests will stay outside for convenience