Remove experiment tracking

[astrojuanlu]

After a discussion in which we considered keeping Experiment Tracking as-is, doing more development work (for example #2152), removal #1831, and making it optional #2079 , with a heavy heart we ended up picking complete removal. There isn't nearly enough user traction and we aren't confident we can make the feature good enough to make people switch over the alternatives.

I think we can proceed without a deprecation warning. This would be a breaking release.

As follow-ups, we'll also hold a retrospective to learn from this experience.

• kedro-datasets: Add deprecation warning for MetricsTrackingDataset and JSONTrackingDataset kedro-plugins#954
• kedro-datasets: remove MetricsTrackingDataset and JSONTrackingDataset kedro#4370
• kedro-starters: remove the experiment tracking setup code kedro#4371
• Remove ET front-end components #2207
• Remove ET back-end components #2208
• Remove ET - Update Documentation and References #2209
• add TSC for approval in the final feature_branch

[tynandebold]

Can I work on this, at least removing the frontend part?

cc @rashidakanchwala

[astrojuanlu]

I have no problem with that @tynandebold!

[astrojuanlu]

@rashidakanchwala will make the call

[Huongg]

After discussing with @tynandebold , here are the three subtasks that need to be completed under this ticket:

• 1. Remove Front-End Components: This includes the UI, e2e tests, and Apollo queries. Estimation: 1–3 points
• 2. Remove Back-End Components: This includes GraphQL Strawberry, the SQL session store, and associated tests. Estimation: 3–5 points
• 3. Update Documentation and References: Remove any references to ET, including Medium articles, YouTube content, and draft a release announcement to address the breaking changes. Estimation: 1–3 points

All should be done in a single feature branch so can be easily used in the future.

[ankatiyar]

While reviewing kedro-org/kedro-starters#262, I was wondering if we need the spaceflights-pandas-viz and the spaceflights-pyspark-viz starters and the "kedro-viz" tool in the project creation flow at all?

The difference between spaceflights-pandas and spaceflights-pyspark-viz is a "reporting" pipeline AND the experiment tracking stuff in the settings.py. Kedro-Viz is in the requirements of all spaceflights starters by default. The "reporting" pipeline is not a part of the project unless you select --tools=viz --example=y. With experiment tracking gone, we could probably get rid of the extra tool and starters? 🤔

[ravi-kumar-pilla]

The difference between spaceflights-pandas and spaceflights-pyspark-viz is a "reporting" pipeline AND the experiment tracking stuff in the settings.py. Kedro-Viz is in the requirements of all spaceflights starters by default. The "reporting" pipeline is not a part of the project unless you select --tools=viz --example=y. With experiment tracking gone, we could probably get rid of the extra tool and starters? 🤔

That is a good call. I think without experiment tracking, we can actually remove both the starters. But it would affect Kedro-Viz visibility too