Add ability to install feature packs on top of existing servers #426

spyrkob · 2023-08-01T14:27:25Z

Support installing features pack on top of already provisioned servers.

Adds prospero features add operation, history and revert support

TomasHofman · 2023-08-02T13:08:45Z

integration-tests/src/test/java/org/wildfly/prospero/it/cli/FeaturesAddTest.java

+        // verify the datasources feature pack was installed successfully
+        final Path modulePath = Path.of("modules", "com", "mysql", "jdbc");
+        assertThat(targetDir.toPath().resolve(modulePath))
+                .exists()
+                .isDirectory();


Just a suggestion - in this test we could check that originally the feature pack is missing, before running the add command.

TomasHofman · 2023-08-02T13:55:44Z

prospero-cli/src/main/java/org/wildfly/prospero/cli/commands/FeaturesCommand.java

+        public Integer call() throws Exception {
+            final long startTime = System.currentTimeMillis();
+
+            if (fpl.split(":").length != 2) {


The version part is not valid input?

No, the version of feature pack is a part of the channel

TomasHofman · 2023-08-02T15:14:11Z

prospero-common/src/main/java/org/wildfly/prospero/actions/FeaturesAddAction.java

+    private final InstallationMetadata metadata;
+    private final ProsperoConfig prosperoConfig;
+    private final Console console;
+    private CandidateActionsFactory candidateActionsFactory;


Can be final too.

prospero-common/src/main/java/org/wildfly/prospero/actions/FeaturesAddAction.java

TomasHofman · 2023-08-02T15:47:54Z

prospero-common/src/main/java/org/wildfly/prospero/actions/PrepareCandidateAction.java

@@ -50,20 +48,19 @@
 class PrepareCandidateAction implements AutoCloseable{


I'm not sure what to imagine under "candidate" term here. Is that just a working installation that maybe gets used at the end of the process?

Yes. The candidate is basically a newly provisioned server with the new feature pack. In next step that candidate is merged with existing server to apply the changes. This is a process that happened in Galleon under the hood, but we explicitly split it up in prospero to allow preparing candidate without stopping the server (in case of update/revert at least).

TomasHofman · 2023-08-03T07:42:13Z

I'm thinking the .provisioning_record.xml should not have the leading dot? It's already in a directory with leading dot, there's no point in hiding it again... It's also inconsistent considering the remaining files in .installation dir.

[thofman@fedora prospero]$ ls eap80/.installation/ -a
.   .cache  installer-channels.yaml  manifest.yaml             README.txt
..  .git    manifest_version.yaml    .provisioning_record.xml

TomasHofman · 2023-08-03T07:51:34Z

That may be out of scope for this PR, but it feels to me that I'm missing some command to review the current state of the installation. I can add a new feature pack but I can't review currently installed feature packs. Something like features list. Then probably also remove.

Maybe I would like to call the command feature-pack and create a shortcut name fp - just thinking about consistent terminology with the maven plugin and docs...

What about some general status command that would display all the important data about the installation? (channel URL or channel version or something, feature packs, model, configs, etc)

TomasHofman · 2023-08-03T08:25:44Z

And a little brainstorming - we could consider something for user to discover available feature packs / models / layers.

TomasHofman · 2023-08-03T08:35:16Z

I think we should also add some paragraph in the docs about this. Correct procedure to follow, since it can require multiple commands to install e.g. the datasource and db driver.

TomasHofman · 2023-08-03T08:38:44Z

BTW I'm trying following:

./prospero install --profile=eap-8.0 --channel=https://.../jboss-set/channel-definitions/-/raw/main/eap-80-test-build/channel.yaml --dir eap80
./prospero features add --dir eap80/ --fpl org.jboss.eap:eap-datasources-galleon-pack --layers datasources-web-server,postgresql-datasource

without creating additional channel and providing driver version. It results in a NPE:

...
Feature-packs resolved.                                                          
Packages installed.               
Downloading artifacts 1/593(0%) activemq-artemis-native-1.0.2.redhat-00003.jarjava.lang.NullPointerException
	at java.base/java.util.Objects.requireNonNull(Objects.java:209)
	at org.wildfly.channel.ArtifactCoordinate.<init>(ArtifactCoordinate.java:33)
	at org.wildfly.prospero.galleon.MavenArtifactMapper.lambda$toChannelArtifacts$0(MavenArtifactMapper.java:59)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
	at org.wildfly.prospero.galleon.MavenArtifactMapper.toChannelArtifacts(MavenArtifactMapper.java:60)
	at org.wildfly.prospero.galleon.ChannelMavenArtifactRepositoryManager.resolveAll(ChannelMavenArtifactRepositoryManager.java:188)
	at org.wildfly.galleon.plugin.WfInstallPlugin.resolveArtifactsInCache(WfInstallPlugin.java:555)
	at org.wildfly.galleon.plugin.WfInstallPlugin.postInstall(WfInstallPlugin.java:410)
	at org.jboss.galleon.runtime.ProvisioningRuntime$3.visitPlugin(ProvisioningRuntime.java:269)
	at org.jboss.galleon.runtime.ProvisioningRuntime$3.visitPlugin(ProvisioningRuntime.java:266)
	at org.jboss.galleon.layout.ProvisioningLayout$Handle.visitPlugins(ProvisioningLayout.java:287)
	at org.jboss.galleon.layout.ProvisioningLayout.visitPlugins(ProvisioningLayout.java:860)
	at org.jboss.galleon.runtime.ProvisioningRuntime.provision(ProvisioningRuntime.java:266)

Update:

I don't have this issue when working with the wildfly-27.0.1 manifest from the intergration-tests dir and the wildfly galleon pack.

spyrkob · 2023-08-03T08:49:13Z

That may be out of scope for this PR, but it feels to me that I'm missing some command to review the current state of the installation. I can add a new feature pack but I can't review currently installed feature packs. Something like features list. Then probably also remove.

That's good idea. I think the status operation would be a good addition for this PR, remove is definately out of scope of this RFE, but I'll create a issue to track it.

Maybe I would like to call the command feature-pack and create a shortcut name fp - just thinking about consistent terminology with the maven plugin and docs...

I like that - @honza-kasik, I think that would affect your tests WDYT?

And a little brainstorming - we could consider something for user to discover available feature packs / models / layers.

Yes there's already #428 to discover available feature packs, and I added #429 to list models and layers. That's something we can add later on though.

I think we should also add some paragraph in the docs about this. Correct procedure to follow, since it can require multiple commands to install e.g. the datasource and db driver.

Yes, absolutely. I need to finish #153 and add it there

spyrkob · 2023-08-08T10:22:34Z

@TomasHofman I added a docs entry. could you take a quick look when you have a moment?

TomasHofman · 2023-08-08T11:02:43Z

dist/docs/guide/usage/add_feature_pack.adoc

+  - groupId: org.wildfly
+    artifactId: wildfly-datasources-galleon-pack
+    version: 4.0.1.Final


Do I remember correctly the datasources galleon pack is going to be part of the main channel?

AFAIK, not in the community version.

dist/docs/guide/usage/add_feature_pack.adoc

honza-kasik · 2023-08-14T08:38:48Z

dist/docs/guide/usage/add_feature_pack.adoc

+
+#### Offline installation
+
+Similarly to other operations, `feature-pack add` can be executed in an offline mode. To do so, users should provide local repositories with all required artifacts (both for base server and the new feature) using `--repositories` parameter.


I'm missing description of --repositories in the feature analysis. Can user also define a channel (with a manifest) for a new feature pack and then use --repositories=<offline-feature-pack-channel-repo>,<offline-base-channel-repo>?

@honza-kasik yes the --repositories work in the same way as in update operation so the user would have to provide both base and feature pack repositories

… add If the provisioning.xml is not recorded during feature add operation, preserve current configuration and use it in revert operations

spyrkob force-pushed the add_features branch from ba7e295 to 8936083 Compare August 1, 2023 14:28

TomasHofman reviewed Aug 2, 2023

View reviewed changes

spyrkob changed the title ~~Add features~~ Add ability to install feature packs on top of existing servers Aug 2, 2023

TomasHofman reviewed Aug 2, 2023

View reviewed changes

spyrkob force-pushed the add_features branch from 2687331 to 540e1ab Compare August 4, 2023 10:55

TomasHofman mentioned this pull request Aug 7, 2023

features list command #433

Closed

TomasHofman approved these changes Aug 8, 2023

View reviewed changes

TomasHofman reviewed Aug 8, 2023

View reviewed changes

dist/docs/guide/usage/add_feature_pack.adoc Outdated Show resolved Hide resolved

spyrkob force-pushed the add_features branch 2 times, most recently from 4e1b51c to 90e72fb Compare August 10, 2023 11:47

honza-kasik reviewed Aug 14, 2023

View reviewed changes

spyrkob force-pushed the add_features branch from 45c5e71 to 2eff55e Compare August 15, 2023 13:46

spyrkob force-pushed the add_features branch 2 times, most recently from 8b8b737 to 09a584f Compare August 23, 2023 08:11

spyrkob force-pushed the add_features branch 2 times, most recently from c0d8ddc to 866b242 Compare September 1, 2023 09:10

spyrkob added 7 commits September 1, 2023 14:27

Add 'features add' command

94f5560

[WFWIP-644] Handle repository override in FeatureAdd action

0c999e2

Code review fixes

6406bc6

Rename features command group to feature-pack

4634253

Handle resolving galleon artifacts without default version

a627962

Persist changes in copy of provisionig.xml after feature add

9fedb3a

Added documentation

47e5122

spyrkob added 8 commits September 1, 2023 14:27

Install default feature pack config if no layers are provided

0497054

Exclude default configs if layers are specified

7856249

Remove obsolete argument from PMU#recordProvisioningDefinition

0ab6ff1

[WFWIP-646] Enable revert for servers provisioned before feature-pack…

a01800c

… add If the provisioning.xml is not recorded during feature add operation, preserve current configuration and use it in revert operations

Verify chosen configuration is available in the provisioned server

1f32e1a

Merge model and config parameters

fec5081

Fix PMD violations

2d41683

Split up default-config and layer config arguments

1f1b804

spyrkob force-pushed the add_features branch 2 times, most recently from 9ded83d to 70a3e7f Compare September 4, 2023 08:54

Remove --default-config option, require --layers

059bc71

spyrkob force-pushed the add_features branch from 70a3e7f to 059bc71 Compare September 6, 2023 11:56

spyrkob merged commit 48545a9 into wildfly-extras:main Sep 7, 2023
5 checks passed

spyrkob deleted the add_features branch September 7, 2023 08:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ability to install feature packs on top of existing servers #426

Add ability to install feature packs on top of existing servers #426

spyrkob commented Aug 1, 2023

TomasHofman Aug 2, 2023 •

edited

Loading

TomasHofman Aug 2, 2023

spyrkob Aug 3, 2023

TomasHofman Aug 2, 2023

TomasHofman Aug 2, 2023 •

edited

Loading

spyrkob Aug 3, 2023

TomasHofman commented Aug 3, 2023 •

edited

Loading

TomasHofman commented Aug 3, 2023

TomasHofman commented Aug 3, 2023

TomasHofman commented Aug 3, 2023

TomasHofman commented Aug 3, 2023 •

edited

Loading

spyrkob commented Aug 3, 2023

spyrkob commented Aug 8, 2023

TomasHofman Aug 8, 2023

spyrkob Aug 8, 2023

honza-kasik Aug 14, 2023

spyrkob Aug 14, 2023

		@@ -50,20 +48,19 @@
		class PrepareCandidateAction implements AutoCloseable{


		#### Offline installation

		Similarly to other operations, `feature-pack add` can be executed in an offline mode. To do so, users should provide local repositories with all required artifacts (both for base server and the new feature) using `--repositories` parameter.

Add ability to install feature packs on top of existing servers #426

Add ability to install feature packs on top of existing servers #426

Conversation

spyrkob commented Aug 1, 2023

TomasHofman Aug 2, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TomasHofman Aug 2, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TomasHofman commented Aug 3, 2023 • edited Loading

TomasHofman commented Aug 3, 2023

TomasHofman commented Aug 3, 2023

TomasHofman commented Aug 3, 2023

TomasHofman commented Aug 3, 2023 • edited Loading

spyrkob commented Aug 3, 2023

spyrkob commented Aug 8, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TomasHofman Aug 2, 2023 •

edited

Loading

TomasHofman Aug 2, 2023 •

edited

Loading

TomasHofman commented Aug 3, 2023 •

edited

Loading

TomasHofman commented Aug 3, 2023 •

edited

Loading