Add support for cluster configuration.

Plan matomo-shared-db now use a 2 nodes Matomo cluster.
Plan dedicated-db now use a Matomo cluster with size between 2 and 10
nodes.
For dedicated plan, the cluster size can be adjusted through service
instance update.

README.md

@@ -48,6 +48,8 @@ Then the default release can also be selected as follows:
 
 If none of `releases.txt` and `default-release.txt` is found at repository root, then by default, only the latest release is proposed for usage and is thus the default and unique one!
 
+Be careful that clean upgrade (especially for production mode) is only supported between releases of the service, due to the support of Liquibase for database update management. There is no guarantee of a clean upgrade from one version to another between releases (i.e., particular commits between two releases). In that case, you should start from an empty database, especially if you want to test the latest version (last commit from the trunk).
+
 ### Setup the service on a CloudFoundry platform
 
 1. Clone this repository and position into the _matomo-cf-release_ directory that has been created.

src/main/doc/markdown/index.md

@@ -14,13 +14,13 @@ Note that the actual releases proposed by the service can be found [here](releas
 Go to your CF marketplace and create a service instance by choosing among the proposed plans. There are three (only the first one is currently implemented, the two others are expected soon). Their objective is to provide different isolation levels in term of load and security of an instance, especially on the management of data (information from tracked Web sites):
 
 1. global-shared-db
-   Data of all instances are stored in a database platform mutualized with many others (useful for dev purpose).
+   Data of all instances are stored in a database platform mutualized with many others (useful for dev purpose). Matomo is run by one container (application instance) within CloudFoundry.
 
 2. matomo-shared-db
-   Data of all instances are stored in a database platform mutualized with all other Matomo service instances of this kind (useful for tracked Web sites with small traffic).
+   Data of all instances are stored in a database platform mutualized with all other Matomo service instances of this kind (useful for tracked Web sites with small / medium traffic). Matomo is run by a cluster of two containers and is configured to run in cluster mode.
 
 3. dedicated-db
-   Data of this Matomo service instance is stored in a dedicated database platform (useful for tracked Web sites with high traffic).
+   Data of this Matomo service instance is stored in a dedicated database platform (useful for tracked Web sites with high traffic). Matomo is run by a cluster of at least two containers and is configured to run in cluster mode. It may scale up two a ten containers cluster as requested by the owner of the service instance.
 
 Indeed, the choice among them depends on the traffic of the tracked Web site and has an impact on the cost of the service instance.
 
@@ -34,6 +34,12 @@ The time zone within which the instance executes can be specified the same way:
 cf cs matomo-service global-shared-db m -c '{"matomoTimeZone": "Europe/Paris"}'
 ```
 
+For the `dedicated-db` plan the number of containers that run the service instance can be specified (by default, it is two of them):
+```
+cf cs matomo-service global-shared-db m -c '{"matomoInstances": 3}'
+```
+It can be ajusted later on through an update action on the instance.
+
 Service upgrade to new release (with higher version) is also supported by the service (see section "Update instance"). Concerning version upgrade, a policy can be specified at creation time:
 ```
 cf cs matomo-service global-shared-db m -c '{"versionUpgradePolicy": "Explicit"}'
@@ -53,7 +59,7 @@ In case the creation of your instance has failed, you can delete it and retry. E
 
 ---
 
-Main instance update actions usually concentrates on changes of plan. This is not currently supported for this service as it requires databse backup/restore. Furthermore, it is not straitforward that such a capability is a strong requirement for this service (to be discussed if needed). Thus, when updating an instance, some parameters need to be specified. Let's go through the different update possibilities using examples. First possibility is to upgrade an instance to a new Matomo release:
+Main instance update actions usually concentrates on changes of plan. This is not currently supported for this service as it requires database backup/restore. Furthermore, it is not straitforward that such a capability is a strong requirement for this service (to be discussed if needed). Thus, when updating an instance, some parameters need to be specified. Let's go through the different update possibilities using examples. First possibility is to upgrade an instance to a new Matomo release:
 ```
 cf update-service m371 -c '{"matomoVersion": "3.10.0"}'
 ```
@@ -62,9 +68,14 @@ The second possibility is to change the version upgrade policy:
 ```
 cf update-service m -c '{"versionUpgradePolicy": "Automatic"}'
 ```
-
 Moving the policy to Automatic means that the instance will be forced to upgrade to the latest release right away.
 
+The third possibility is to adjust the number of containers that run the instance (only in case of `dedicated-db plan`):
+```
+cf update-service m -c '{"matomoInstances": 6}'
+```
+This number is forced to stay in the interval [2..10]. This means that if a lower value than 2 is specified, then 2 is forced. In the same way, if a higher value than 10 is specified, then 10 is forced.
+
 ## Bind to instances
 
 ---

src/main/java/com/orange/oss/matomocfservice/cfmgr/CloudFoundryMgr.java

@@ -19,6 +19,7 @@
 import java.io.File;
 import java.io.IOException;
 import java.nio.file.Files;
+import java.nio.file.Path;
 import java.nio.file.Paths;
 import java.nio.file.StandardOpenOption;
 import java.util.ArrayList;
@@ -35,6 +36,7 @@
 import org.cloudfoundry.operations.applications.GetApplicationRequest;
 import org.cloudfoundry.operations.applications.PushApplicationManifestRequest;
 import org.cloudfoundry.operations.applications.Route;
+import org.cloudfoundry.operations.applications.ScaleApplicationRequest;
 import org.cloudfoundry.operations.services.GetServiceInstanceRequest;
 import org.cloudfoundry.operations.services.ServiceInstance;
 import org.cloudfoundry.operations.services.UnbindServiceInstanceRequest;
@@ -171,7 +173,7 @@ public String getAppUrlPrefix(String appcode) {
 	 * @param instid	The code name of the instance
 	 * @return	The Mono to signal the end of the async process (produce nothng indeed)
 	 */
-	public Mono<Void> deployMatomoCfApp(String instid, String version, String expohost, String planid, String tz) {
+	public Mono<Void> deployMatomoCfApp(String instid, String version, String expohost, String planid, String tz, int memsize, int nbinst) {
 		LOGGER.debug("CFMGR::deployMatomoCfApp: instId={}", instid);
 		String instpath = matomoReleases.getVersionPath(version, instid);
 		LOGGER.debug("File for Matomo bits: " + instpath);
@@ -182,16 +184,17 @@ public Mono<Void> deployMatomoCfApp(String instid, String version, String expoho
 					.path(Paths.get(instpath))
 					.route(Route.builder().route(getHost(instid, expohost) + "." + properties.getDomain()).build())
 					.buildpack(properties.getPhpBuildpack())
-					.memory(256)
+					.memory(memsize)
 					.timeout(180)
+					.instances(nbinst)
 					.environmentVariable("TZ", tz);
 		} else {
 			manifestbuilder = ApplicationManifest.builder()
 					.name(getAppName(instid))
 					.path(Paths.get(instpath))
 					.route(Route.builder().route(getHost(instid, expohost) + "." + properties.getDomain()).build())
 					.buildpack(properties.getPhpBuildpack())
-					.memory(256)
+					.memory(memsize)
 					.timeout(180);			
 		}
 		List<String> services = new ArrayList<String>();
@@ -203,6 +206,13 @@ public Mono<Void> deployMatomoCfApp(String instid, String version, String expoho
 				.build());
 	}
 
+	public Mono<Void> scaleMatomoCfApp(String instid, int instances) {
+		return cfops.applications().scale(ScaleApplicationRequest.builder()
+				.name(getAppName(instid))
+				.instances(instances)
+				.build());
+	}
+
 	/**
 	 * Launch the creation of a dedicated DB for a particular Matomo instance.
 	 * @param instid	The code name of the instance
@@ -292,8 +302,8 @@ public Mono<Void> deleteMatomoCfApp(String instid, String planid) {
 				});
 	}
 
-	public Mono<AppConfHolder> getInstanceConfigFile(String instid, String version) {
-		LOGGER.debug("CFMGR::getInstanceConfigFile: instid={}, version={}", instid, version);
+	public Mono<AppConfHolder> getInstanceConfigFile(String instid, String version, boolean clustermode) {
+		LOGGER.debug("CFMGR::getInstanceConfigFile: instid={}, version={}, clusterMode={}", instid, version, clustermode);
 		AppConfHolder appidh = new AppConfHolder();
 		return Mono.create(sink -> {
 			cfops.applications().get(GetApplicationRequest.builder().name(getAppName(instid)).build())
@@ -313,8 +323,21 @@ public Mono<AppConfHolder> getInstanceConfigFile(String instid, String version)
 						ssh.authPassword("cf:" + appidh.appId + "/0", pwd);
 						String target = matomoReleases.getVersionPath(version, instid) + File.separator + "config" + File.separator;
 						ssh.newSCPFileTransfer().download("/home/vcap/app/htdocs/config/config.ini.php", new FileSystemFile(target));
+						Path pcf = Paths.get(target + "config.ini.php");
 						if (properties.getMatomoDebug()) {
-							Files.write(Paths.get(target + "config.ini.php"), "\n[Tracker]\ndebug = 1\nenable_sql_profiler = 1\n".getBytes(), StandardOpenOption.APPEND);
+							Files.write(pcf, "\n[Tracker]\ndebug = 1\nenable_sql_profiler = 1\n".getBytes(), StandardOpenOption.APPEND);
+						}
+						if (clustermode) {
+							List<String> allLines = Files.readAllLines(pcf);
+							Files.write(pcf, "".getBytes());
+							for (String line : allLines) {
+								Files.write(pcf, (line + "\n").getBytes(), StandardOpenOption.APPEND);
+								if (line.startsWith("[General]")) {
+									LOGGER.debug("CFMGR::getInstanceConfigFile: add configuration for cluster mode support");
+									Files.write(pcf, "session_save_handler = dbtable\n".getBytes(), StandardOpenOption.APPEND);
+									Files.write(pcf, "multi_server_environment = 1\n".getBytes(), StandardOpenOption.APPEND);									
+								}
+							}
 						}
 						appidh.fileContent = Files.readAllBytes(Paths.get(target + "config.ini.php"));
 						sink.success(appidh);

src/main/java/com/orange/oss/matomocfservice/web/domain/PMatomoInstance.java

@@ -24,6 +24,7 @@
 
 import com.orange.oss.matomocfservice.api.model.OpCode;
 import com.orange.oss.matomocfservice.api.model.PlatformKind;
+import com.orange.oss.matomocfservice.config.ServiceCatalogConfiguration;
 
 /**
  * @author P. Déchamboux
@@ -77,6 +78,8 @@ public class PMatomoInstance extends POperationStatus {
 	private byte[] configFileContent;
 
 	private boolean automaticVersionUpgrade;
+
+	private int instances;
 
 	@SuppressWarnings("unused")
 	protected PMatomoInstance() {
@@ -94,9 +97,10 @@ protected PMatomoInstance() {
 		this.tokenAuth = null;
 		this.configFileContent = null;
 		this.automaticVersionUpgrade = true;
+		this.instances = 1;
 	}
 
-	public PMatomoInstance(String id, int idUrl, String servDefId, String name, PlatformKind pfkind, String pfapi, String planid, PPlatform pf, String version, boolean avu) {
+	public PMatomoInstance(String id, int idUrl, String servDefId, String name, PlatformKind pfkind, String pfapi, String planid, PPlatform pf, String version, boolean avu, int instances) {
 		super(id, OpCode.CREATE.toString(), OperationState.IN_PROGRESS.getValue(), pf);
 		this.idUrl = idUrl;
 		this.serviceDefinitionId = servDefId;
@@ -109,6 +113,7 @@ public PMatomoInstance(String id, int idUrl, String servDefId, String name, Plat
 		this.tokenAuth = null;
 		this.configFileContent = null;
 		this.automaticVersionUpgrade = avu;
+		this.instances = instances;
 	}
 
 	public int getIdUrl() {
@@ -141,7 +146,14 @@ public String getPlatformApiLocation() {
 	}
 
 	public String getPlanId() {
-		return planId;
+		return this.planId;
+	}
+
+	public boolean getClusterMode() {
+		if (this.planId.equals(ServiceCatalogConfiguration.PLANGLOBSHARDB_UUID)) {
+			return false;
+		}
+		return true;
 	}
 
 	public byte[] getConfigFileContent() {
@@ -162,6 +174,15 @@ public void setAutomaticVersionUpgrade(boolean avu) {
 		super.touch();
 	}
 
+	public int getInstances() {
+		return this.instances;
+	}
+
+	public void setIntances(int instances) {
+		this.instances = instances;
+		super.touch();
+	}
+
 	public void setInstalledVersion(String instVers) {
 		this.installedVersion = instVers;
 		super.touch();