Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PoC - independent solution repo structure #179710

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

lukeelmers
Copy link
Member

@lukeelmers lukeelmers commented Mar 29, 2024

Definitely don't merge this

This is a proof-of-concept which explores what it would look like to make solutions more independent by restructuring the repo in a way that enforces new plugin-level boundaries. These boundaries prevent solution plugins from having dependencies on each other, forcing any shared dependencies to live in the platform:
Screenshot 2024-05-30 at 11 27 47 AM

These boundaries are enforced by the directory structure of the repo. For illustrative purposes, this is the directory structure used in this PR (the final structure would likely be a bit different):

.
├── src
│   └── platform
│       └── internal // not available to plugins outside of platform
└── x-pack
    ├── observability
    ├── platform
    │   └── internal // not available to plugins outside of platform
    ├── search
    └── security

Plugins are only allowed to have dependencies on plugins which exist in the platform directory, so for example, a plugin in security could not depend on another in observability.

Plugins which are leftover in src/plugins and x-pack/plugins were identified as problematic because they don't currently adhere to these restrictions and would need to be refactored or have bits relocated to platform. These include:

ai_assistant_management/observability
x-pack/enterprise_search
x-pack/index_lifecycle_management
x-pack/ingest_pipelines
x-pack/monitoring_collection
x-pack/observability_solution/infra
x-pack/observability_solution/logs_shared
x-pack/osquery
x-pack/upgrade_assistant

The enforcement logic in 7077867 may not be how we'd actually implement this in production, especially if we were looking to add similar logic to packages... in that case it would be worth considering strategies for making this more generalized.

Follow-up tasks not done here

  • move src/core -> src/platform/core (requires a lot of changes to hard-coded paths)
  • split up packages & evaluate options for enforcement (kbn_references?)
  • look for easier ways to update paths in config files (can this be automated without too much effort?)
  • split up global eslint config and relocate pieces in solution directories
  • split up FTR configs and relocate to relevant solution directories

@lukeelmers

This comment was marked as outdated.

1 similar comment
@lukeelmers

This comment was marked as outdated.

@lukeelmers lukeelmers force-pushed the poc/solution-boundaries branch from 3c0f4a6 to cdb2f2a Compare March 29, 2024 19:33
@lukeelmers

This comment was marked as outdated.

@lukeelmers lukeelmers force-pushed the poc/solution-boundaries branch from cdb2f2a to 0e83e3b Compare March 29, 2024 19:43
@lukeelmers

This comment was marked as outdated.

@lukeelmers

This comment was marked as outdated.

@lukeelmers

This comment was marked as outdated.

4 similar comments
@lukeelmers

This comment was marked as outdated.

@lukeelmers

This comment was marked as outdated.

@lukeelmers

This comment was marked as outdated.

@lukeelmers

This comment was marked as outdated.

@lukeelmers lukeelmers force-pushed the poc/solution-boundaries branch 2 times, most recently from 34faf4f to f54bcbe Compare April 2, 2024 18:06
@lukeelmers

This comment was marked as outdated.

@lukeelmers lukeelmers force-pushed the poc/solution-boundaries branch from f54bcbe to 33bf421 Compare April 2, 2024 18:33
@lukeelmers

This comment was marked as outdated.

@lukeelmers lukeelmers force-pushed the poc/solution-boundaries branch from 33bf421 to 418b13c Compare April 2, 2024 19:04
@lukeelmers

This comment was marked as outdated.

@lukeelmers lukeelmers force-pushed the poc/solution-boundaries branch from 340afc5 to 8e665d0 Compare April 2, 2024 21:30
@lukeelmers

This comment was marked as outdated.

4 similar comments
@lukeelmers

This comment was marked as outdated.

@lukeelmers

This comment was marked as outdated.

@lukeelmers

This comment was marked as outdated.

@lukeelmers

This comment was marked as outdated.

@lukeelmers lukeelmers force-pushed the poc/solution-boundaries branch from 193c5b5 to 1f290a9 Compare May 15, 2024 02:03
@lukeelmers

This comment was marked as outdated.

@kibana-ci
Copy link
Collaborator

kibana-ci commented May 15, 2024

💔 Build Failed

Failed CI Steps

History

  • 💔 Build #209990 failed 193c5b5afb6a8831f433cf17ee3bd86623792195
  • 💔 Build #201270 failed a23cf7216044e9ee44803d3a98da8c002cc7dff7
  • 💔 Build #201173 failed 07821f7698526d8379ddc7c04f508a70529c27ce
  • 💔 Build #200964 failed da982d779495193d64595fd4415284f78f0929ff
  • 💔 Build #200697 failed 8e665d023b6fe10cd89d70863d9e19cf989f5167

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@lukeelmers lukeelmers force-pushed the poc/solution-boundaries branch from 1f290a9 to 7077867 Compare May 15, 2024 02:42
gsoldevila added a commit that referenced this pull request Oct 25, 2024
## Summary

Follow-up of #195367
As part of the Sustainable Kibana Architecture initiative, this PR
leverages the mechanisms and concepts introduced in
#194810, updating plugins that
were considered to be solution-specific in Luke's
[PoC](#179710).

This might trigger linting rule violations in CI, and help uncover
conflicts related to forbidden dependencies.
As soon as they are resolved, we can proceed to classify solutions'
plugins.
gsoldevila added a commit to gsoldevila/kibana that referenced this pull request Nov 7, 2024
…95375)

## Summary

Follow-up of elastic#195367
As part of the Sustainable Kibana Architecture initiative, this PR
leverages the mechanisms and concepts introduced in
elastic#194810, updating plugins that
were considered to be solution-specific in Luke's
[PoC](elastic#179710).

This might trigger linting rule violations in CI, and help uncover
conflicts related to forbidden dependencies.
As soon as they are resolved, we can proceed to classify solutions'
plugins.

(cherry picked from commit a5517d9)

# Conflicts:
#	src/plugins/maps_ems/kibana.jsonc
#	x-pack/plugins/file_upload/kibana.jsonc
#	x-pack/plugins/maps/kibana.jsonc
gsoldevila added a commit to gsoldevila/kibana that referenced this pull request Nov 7, 2024
…95375)

## Summary

Follow-up of elastic#195367
As part of the Sustainable Kibana Architecture initiative, this PR
leverages the mechanisms and concepts introduced in
elastic#194810, updating plugins that
were considered to be solution-specific in Luke's
[PoC](elastic#179710).

This might trigger linting rule violations in CI, and help uncover
conflicts related to forbidden dependencies.
As soon as they are resolved, we can proceed to classify solutions'
plugins.

(cherry picked from commit a5517d9)

# Conflicts:
#	src/plugins/maps_ems/kibana.jsonc
#	x-pack/plugins/file_upload/kibana.jsonc
#	x-pack/plugins/maps/kibana.jsonc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants