Pal 1027 palisade shell

NOTICES.md

@@ -4,6 +4,8 @@ List of third-party dependencies grouped by their license type
 * Jackson-annotations ([com.fasterxml.jackson.core:jackson-annotations:2.11.0](http://github.com/FasterXML/jackson))
 * jackson-databind ([com.fasterxml.jackson.core:jackson-databind:2.10.0](http://github.com/FasterXML/jackson))
 * Jackson datatype: jdk8 ([com.fasterxml.jackson.datatype:jackson-datatype-jdk8:2.10.0](https://github.com/FasterXML/jackson-modules-java8/jackson-datatype-jdk8))
+* akka-http ([com.typesafe.akka:akka-http_2.13:10.2.1](https://akka.io))
+* akka-stream ([com.typesafe.akka:akka-stream_2.13:2.6.10](https://akka.io/))
 * Micronaut ([io.micronaut:micronaut-http-server-netty:2.3.2](http://micronaut.io))
 * Micronaut ([io.micronaut:micronaut-inject:2.3.2](http://micronaut.io))
 * Micronaut ([io.micronaut:micronaut-inject-java:2.3.2](http://micronaut.io))
@@ -12,16 +14,16 @@ List of third-party dependencies grouped by their license type
 * RxJava ([io.reactivex.rxjava3:rxjava:3.0.8](https://github.com/ReactiveX/RxJava))
 * AssertJ fluent assertions ([org.assertj:assertj-core:3.19.0](https://assertj.github.io/doc/assertj-core/))
 * org.immutables.value ([org.immutables:value:2.8.2](http://immutables.org/value))
+* Spring Boot AOP Starter ([org.springframework.boot:spring-boot-starter-aop:2.2.0.RELEASE](https://projects.spring.io/spring-boot/#/spring-boot-parent/spring-boot-starters/spring-boot-starter-aop))
+* Spring Boot Test Starter ([org.springframework.boot:spring-boot-starter-test:2.2.0.RELEASE](https://projects.spring.io/spring-boot/#/spring-boot-parent/spring-boot-starters/spring-boot-starter-test))
+* Spring Shell Starter ([org.springframework.shell:spring-shell-starter:2.0.0.RELEASE](http://projects.spring.io/spring-boot/spring-shell-parent/spring-shell-starter/))
+* client-java ([uk.gov.gchq.palisade:client-java:0.5.0-SNAPSHOT](https://github.com/gchq/Palisade-clients/tree/develop/client-java))
 * common ([uk.gov.gchq.palisade:common:0.5.0-SNAPSHOT](https://github.com/gchq/Palisade-common))
 
-### [Apache-2.0](./licenses/not recognised):
-* akka-http ([com.typesafe.akka:akka-http_2.13:10.2.1](https://akka.io))
-* akka-stream ([com.typesafe.akka:akka-stream_2.13:2.6.10](https://akka.io/))
-
 ### [Eclipse Public License 1.0](./licenses/eclipse_public_license_1.0.html):
 * Logback Classic Module ([ch.qos.logback:logback-classic:1.2.3](http://logback.qos.ch/logback-classic))
 
-### [Eclipse Public License v2.0](./licenses/not recognised):
+### [Eclipse Public License 2.0](./licenses/eclipse_public_license_2.0.html):
 * JUnit Jupiter (Aggregator) ([org.junit.jupiter:junit-jupiter:5.7.0](https://junit.org/junit5/))
 * JUnit Jupiter API ([org.junit.jupiter:junit-jupiter-api:5.7.0](https://junit.org/junit5/))
 * JUnit Jupiter Engine ([org.junit.jupiter:junit-jupiter-engine:5.7.0](https://junit.org/junit5/))
@@ -32,6 +34,7 @@ List of third-party dependencies grouped by their license type
 * Logback Classic Module ([ch.qos.logback:logback-classic:1.2.3](http://logback.qos.ch/logback-classic))
 
 ### [MIT License](./licenses/mit_license.txt):
+* jnr-fuse ([com.github.serceman:jnr-fuse:0.5.5](https://github.com/SerCeMan/jnr-fuse))
 * mockito-core ([org.mockito:mockito-core:3.7.7](https://github.com/mockito/mockito))
 * mockito-junit-jupiter ([org.mockito:mockito-junit-jupiter:3.7.7](https://github.com/mockito/mockito))
 * SLF4J API Module ([org.slf4j:slf4j-api:1.7.26](http://www.slf4j.org))

client-fuse/README.md

@@ -0,0 +1,54 @@
+<!---
+Copyright 2018-2021 Crown Copyright
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+--->
+
+# Palisade FuseFS Client
+
+## Introduction
+> ***Filesystem in Userspace (FUSE)*** is a software interface for Unix and Unix-like computer operating systems that lets non-privileged users create their own file systems without editing kernel code.
+This is achieved by running file system code in user space while the FUSE module provides only a "bridge" to the actual kernel interfaces.
+[[1]](https://en.wikipedia.org/wiki/Filesystem_in_Userspace)
+
+Within the context of Palisade, this allows us to make a request with `userId`, `resourceId` and `context`, then create a software-controlled filesystem mount to represent the returned resources and data.
+The Filtered-Resource Service response represents a directory listing (we must convert it into a tree and perform some simple processing).
+The Data Service read represents a file read.
+
+
+## Requirements
+
+* [libfuse for Linux and BSD](https://github.com/libfuse/libfuse)
+    * [libfuse3-dev for Ubuntu](https://packages.ubuntu.com/hirsute/libfuse3-dev)
+    * [libfuse3-dev for Debian](https://packages.debian.org/sid/libfuse3-dev)
+    * [fuse2 for Arch](https://archlinux.org/packages/extra/x86_64/fuse2/)
+* [OSXFUSE for MacOS](https://osxfuse.github.io/)
+* [WinFSP for Windows](http://www.secfs.net/winfsp/)
+
+
+## Usage
+
+Register a request to Palisade and the `<mountDir>` will be populated with resources.
+```shell script
+java -jar client-fuse.jar <clientUri> <resourceId> <mountDir>
+```
+See the [URL configuration](../client-java/README.md#URL) and [client properties](../client-java/README.md#Client%20properties) sections of the [client-java README](../client-java/README.md) for details on the `<clientUri>`.
+
+
+## Notes
+
+In terms of ease-of-use and approachability, FUSE is an easy way to get started with Palisade.
+All existing tools for working with UNIX filesystems and directory trees will work without any changes required.
+
+Despite this, this is not particularly flexible (no option for exposing additional resource metadata), memory-efficient (the full tree of returned resources must be stored), or scalable (not appropriate for a distributed map-reduce scenario).
+A 'power-user' may prefer working directly with the [Java Client](../client-java/README.md).

client-fuse/mvn_dependency_tree.txt

@@ -0,0 +1,39 @@
+uk.gov.gchq.palisade:client-fuse:jar:0.5.0-SNAPSHOT
++- com.github.serceman:jnr-fuse:jar:0.5.5:compile
+|  +- com.github.jnr:jnr-ffi:jar:2.1.12:compile
+|  |  +- com.github.jnr:jffi:jar:1.2.23:compile
+|  |  +- com.github.jnr:jffi:jar:native:1.2.23:runtime
+|  |  +- org.ow2.asm:asm:jar:7.1:compile
+|  |  +- org.ow2.asm:asm-commons:jar:7.1:compile
+|  |  +- org.ow2.asm:asm-analysis:jar:7.1:compile
+|  |  +- org.ow2.asm:asm-tree:jar:7.1:compile
+|  |  +- org.ow2.asm:asm-util:jar:7.1:compile
+|  |  +- com.github.jnr:jnr-a64asm:jar:1.0.0:compile
+|  |  \- com.github.jnr:jnr-x86asm:jar:1.0.2:compile
+|  +- com.github.jnr:jnr-posix:jar:3.0.54:compile
+|  \- com.github.jnr:jnr-constants:jar:0.9.15:compile
++- uk.gov.gchq.palisade:client-java:jar:0.5.0-SNAPSHOT:compile
+|  +- uk.gov.gchq.palisade:common:jar:0.5.0-SNAPSHOT:compile
+|  |  \- org.apache.avro:avro:jar:1.8.2:compile
+|  |     +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
+|  |     +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
+|  |     +- com.thoughtworks.paranamer:paranamer:jar:2.7:compile
+|  |     +- org.xerial.snappy:snappy-java:jar:1.1.1.3:compile
+|  |     +- org.apache.commons:commons-compress:jar:1.8.1:compile
+|  |     \- org.tukaani:xz:jar:1.5:compile
+|  +- org.slf4j:slf4j-api:jar:1.7.28:compile
+|  +- com.fasterxml.jackson.core:jackson-databind:jar:2.10.0:compile
+|  |  +- com.fasterxml.jackson.core:jackson-annotations:jar:2.10.0:compile
+|  |  \- com.fasterxml.jackson.core:jackson-core:jar:2.10.0:compile
+|  +- com.fasterxml.jackson.datatype:jackson-datatype-jdk8:jar:2.10.0:compile
+|  \- io.reactivex.rxjava3:rxjava:jar:3.0.8:compile
+|     \- org.reactivestreams:reactive-streams:jar:1.0.3:compile
++- org.junit.jupiter:junit-jupiter:jar:5.7.0:test
+|  +- org.junit.jupiter:junit-jupiter-api:jar:5.5.2:test
+|  |  +- org.apiguardian:apiguardian-api:jar:1.1.0:test
+|  |  +- org.opentest4j:opentest4j:jar:1.2.0:test
+|  |  \- org.junit.platform:junit-platform-commons:jar:1.5.2:test
+|  +- org.junit.jupiter:junit-jupiter-params:jar:5.5.2:test
+|  \- org.junit.jupiter:junit-jupiter-engine:jar:5.5.2:test
+|     \- org.junit.platform:junit-platform-engine:jar:1.5.2:test
+\- org.assertj:assertj-core:jar:3.19.0:test

client-fuse/pom.xml

@@ -0,0 +1,70 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+  ~ Copyright 2018-2021 Crown Copyright
+  ~
+  ~ Licensed under the Apache License, Version 2.0 (the "License");
+  ~ you may not use this file except in compliance with the License.
+  ~ You may obtain a copy of the License at
+  ~
+  ~     http://www.apache.org/licenses/LICENSE-2.0
+  ~
+  ~ Unless required by applicable law or agreed to in writing, software
+  ~ distributed under the License is distributed on an "AS IS" BASIS,
+  ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  ~ See the License for the specific language governing permissions and
+  ~ limitations under the License.
+  -->
+
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+
+    <parent>
+        <groupId>uk.gov.gchq.palisade</groupId>
+        <artifactId>clients</artifactId>
+        <version>0.5.0-${revision}</version>
+        <relativePath>../pom.xml</relativePath>
+    </parent>
+    <modelVersion>4.0.0</modelVersion>
+
+    <!--  *********** Artifact ID ***********    -->
+    <artifactId>client-fuse</artifactId>
+    <url>https://github.com/gchq/Palisade-clients/tree/develop/client-fuse</url>
+    <!--  *********** Artifact ID ***********    -->
+
+    <properties>
+        <java.version>11</java.version>
+        <maven.compiler.source>11</maven.compiler.source>
+        <maven.compiler.target>11</maven.compiler.target>
+
+        <junit.jupiter.version>5.7.0</junit.jupiter.version>
+        <assertj.version>3.19.0</assertj.version>
+    </properties>
+
+    <dependencies>
+        <dependency>
+            <groupId>com.github.serceman</groupId>
+            <artifactId>jnr-fuse</artifactId>
+            <version>0.5.5</version>
+        </dependency>
+        <dependency>
+            <groupId>uk.gov.gchq.palisade</groupId>
+            <artifactId>client-java</artifactId>
+            <version>0.5.0-SNAPSHOT</version>
+        </dependency>
+
+        <dependency>
+            <groupId>org.junit.jupiter</groupId>
+            <artifactId>junit-jupiter</artifactId>
+            <version>${junit.jupiter.version}</version>
+            <scope>test</scope>
+        </dependency>
+        <dependency>
+            <groupId>org.assertj</groupId>
+            <artifactId>assertj-core</artifactId>
+            <version>${assertj.version}</version>
+            <scope>test</scope>
+        </dependency>
+    </dependencies>
+
+</project>

client-fuse/src/main/java/uk/gov/gchq/palisade/client/fuse/FuseClient.java

@@ -0,0 +1,63 @@
+/*
+ * Copyright 2018-2021 Crown Copyright
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package uk.gov.gchq.palisade.client.fuse;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import uk.gov.gchq.palisade.client.fuse.client.ResourceTreeClient;
+import uk.gov.gchq.palisade.client.fuse.client.ResourceTreeClient.ResourceTreeWithContext;
+import uk.gov.gchq.palisade.client.fuse.fs.ResourceTreeFS;
+import uk.gov.gchq.palisade.client.fuse.tree.impl.LeafResourceNode;
+import uk.gov.gchq.palisade.client.internal.dft.DefaultClient;
+
+import java.io.InputStream;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.function.Function;
+
+public class FuseClient {
+    private static final Logger LOGGER = LoggerFactory.getLogger(FuseClient.class);
+    private final ResourceTreeClient client;
+
+    public FuseClient(final String clientUri) {
+        this.client = new ResourceTreeClient(new DefaultClient().connect(clientUri));
+    }
+
+    public static void main(final String... args) {
+        if (args.length == 4) {
+            new FuseClient(args[1])
+                    .mount(args[2], args[3]);
+        } else {
+            LOGGER.error("Usage: {} <clientUri> <resourceId> <mountDir>", args[0]);
+        }
+    }
+
+    public void mount(final String resourceId, final String mountDir) {
+        Path mountPath = Paths.get(mountDir);
+
+        ResourceTreeWithContext tree = client.register(resourceId);
+        Function<LeafResourceNode, InputStream> reader = node -> client.read(tree.getToken(), node);
+        ResourceTreeFS fuseFs = new ResourceTreeFS(tree, reader);
+
+        try {
+            fuseFs.mount(mountPath, true, false);
+        } finally {
+            fuseFs.umount();
+        }
+    }
+}

client-fuse/src/main/java/uk/gov/gchq/palisade/client/fuse/client/LeafResourceQueryItem.java

@@ -0,0 +1,50 @@
+/*
+ * Copyright 2018-2021 Crown Copyright
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package uk.gov.gchq.palisade.client.fuse.client;
+
+import uk.gov.gchq.palisade.client.QueryItem;
+import uk.gov.gchq.palisade.resource.LeafResource;
+
+class LeafResourceQueryItem implements QueryItem {
+    private final LeafResource leafResource;
+    private final String token;
+
+    protected LeafResourceQueryItem(final LeafResource leafResource, final String token) {
+        this.leafResource = leafResource;
+        this.token = token;
+    }
+
+    @Override
+    public ItemType getType() {
+        return ItemType.RESOURCE;
+    }
+
+    @Override
+    public String getToken() {
+        return token;
+    }
+
+    @Override
+    public String asError() {
+        return null;
+    }
+
+    @Override
+    public LeafResource asResource() {
+        return leafResource;
+    }
+}

client-fuse/src/main/java/uk/gov/gchq/palisade/client/fuse/client/OnNextStubSubscriber.java

@@ -0,0 +1,31 @@
+/*
+ * Copyright 2018-2021 Crown Copyright
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package uk.gov.gchq.palisade.client.fuse.client;
+
+import java.util.concurrent.Flow.Subscriber;
+import java.util.concurrent.Flow.Subscription;
+
+public interface OnNextStubSubscriber<T> extends Subscriber<T> {
+    default void onSubscribe(final Subscription subscription) {
+    }
+
+    default void onError(final Throwable throwable) {
+    }
+
+    default void onComplete() {
+    }
+}