Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pal 1027 palisade shell #58

Merged
merged 23 commits into from
Apr 30, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
475159c
PAL-1027-palisade-shell initial commit
dev930018 Apr 6, 2021
77c09d9
PAL-1027 client-shell and client-fuse modules
dev930018 Apr 6, 2021
9846309
PAL-1027 fuse using java client
dev930018 Apr 7, 2021
36c921b
PAL-1027 Client-fuse and Client-shell docs and licenses
dev930018 Apr 13, 2021
e50f803
Merge branch 'develop' of https://github.com/gchq/Palisade-clients in…
dev930018 Apr 21, 2021
9389e2d
PAL-1027 working shell client with examples in docs
dev930018 Apr 21, 2021
f68e981
PAL-1027 removed unused epl license
dev930018 Apr 21, 2021
f4a319b
PAL-1027 added timeout during testing
dev930018 Apr 21, 2021
9d7e130
PAL-1027 normalized module names for global test configuration
dev930018 Apr 21, 2021
e02ff5c
PAL-1027 parameterised dependency versions
dev930018 Apr 21, 2021
1589016
PAL-1027 requirements docs tidying
dev930018 Apr 26, 2021
7affaa2
PAL-1027 addressed PR feedback for README, pom and AOP class location
dev930018 Apr 28, 2021
0c5240d
PAL-1027 some documentation and some fuse xattr bits
dev930018 Apr 28, 2021
ea5a8f2
PAL-1027 client-fuse documentation
dev930018 Apr 29, 2021
5c7eec9
PAL-1027 self-closing <a /> field
dev930018 Apr 29, 2021
51f6e01
PAL-1027 missing close-quote and @implNote
dev930018 Apr 29, 2021
c0af362
PAL-1027 documentation and sq fixes
dev930018 Apr 29, 2021
e0e6d8b
PAL-1027 removed illegal > in javadoc
dev930018 Apr 29, 2021
6e9ec1b
PAL-1027 more sq smells on statics and copies
dev930018 Apr 29, 2021
61767f4
PAL-1027 document runtime io exception constructor
dev930018 Apr 30, 2021
9b60504
Merge branch 'develop' into PAL-1027-palisade-shell
dev930018 Apr 30, 2021
217071c
PAL-1027 pr feedback and assertj .as(..) descriptions
dev930018 Apr 30, 2021
a14c111
PAL-1027 missing .as(..)
dev930018 Apr 30, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,8 @@ public FuseClient(final String clientUri) {
/**
* Run a simple CLI application for this client, taking the clientUri, resourceId and mountDir from command-line args
*
* @param args command-line arguments, expected to contain the application name and three passed arguments
* @param args Command-line arguments, expected to contain the application name and three-or-more passed arguments.
* The args list should be ordered [this-jar-name, client-uri-config, resource-id, local-mount-point, context-map...]
* e.g. [client-fuse.jar, pal://cluster/?userid=Alice, file:/data/local-data-store/, /mnt/palisade, purpose=SALARY]
dev930018 marked this conversation as resolved.
Show resolved Hide resolved
*/
public static void main(final String... args) {
Expand Down Expand Up @@ -94,7 +95,7 @@ public static void main(final String... args) {
}

/**
* Register a request with palisade using a configured client.
* Register a request with Palisade using a configured client.
* Mount the fuse directory and block until application exit.
* Attempt to gracefully unmount on application exit.
*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ public static class ResourceTreeWithContext extends ResourceTree {
private final String token;

/**
* Construct a new ResourceTree and keep track of the given token.
* Construct a new ResourceTree and keep track of the given token.
*
* @param token the response from the Palisade Service used to identify this
* request with the Data Service.
Expand Down Expand Up @@ -114,11 +114,11 @@ protected Resource reapplyScheme(final Resource resource) {
}

/**
* Substitute a data-service address using a env map of connection-detail
* Substitute a Data Service address using a env map of connection-detail
* service names to their substitutions.
* If a serviceName is not found in the map, it is unchanged.
*
* @param env the map of data-service names to substitutions
* @param env the map of Data Service names to substitutions
* @param serviceName the service-name to lookup in the map
* @return the substitution for this service-name
*/
Expand All @@ -139,6 +139,14 @@ protected UnaryOperator<Resource> substDataServiceAddress(final Map<String, Stri
};
}

/**
* Apply a formatting function (unary operator) to a resource and each of its parents recursively.
*
* @param resource the resource to apply the formatter function to (it will also apply to all parents)
* @param formatter some function that will edit each of the resources, eg. adding a prefix to the id
* @param <T> the type of the resource
* @return the resource after applying the formatting function to it and its parents
*/
protected <T extends Resource> T formatResource(final T resource, final UnaryOperator<Resource> formatter) {
dev930018 marked this conversation as resolved.
Show resolved Hide resolved
if (resource instanceof ChildResource) {
((ChildResource) resource).parent(formatResource(((ChildResource) resource).getParent(), formatter));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,9 @@ public ResourceTreeFS(final ResourceTree resourceTree, final Function<LeafResour
}

private static int getattr(final TreeNode<Resource> node, final FuseContext ctx, final FileStat stat) {
final int readOnly = FileStat.S_IRUSR | FileStat.S_IRGRP; // ug+r
// Mark the file as read-only to the owning user and group, no access to others
// Same as "chmod ug=r o=", "chmod 440", or an "ls" permissions line of "r--r-----"
final int readOnly = FileStat.S_IRUSR | FileStat.S_IRGRP;
if (node instanceof ParentNode) {
// Directory, read-only
stat.st_mode.set(FileStat.S_IFDIR | readOnly);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ private static Optional<TreeNode<Resource>> getNode(final TreeNode<Resource> nod
}
}

/* Collections API */
/* Implementations for Collection interface */

@Override
public int size() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,11 +52,11 @@ public class BranchResourceNode implements ParentNode<Resource>, ChildNode<Resou
// We actively want the parent to be a mutable ref, not a copy
@SuppressWarnings("java:S2384")
public BranchResourceNode(final String id, final ParentNode<ParentResource> parent, final ChildResource resource) {
this.id = id;
this.parent = parent;
if (!(resource instanceof ParentResource)) {
throw new IllegalArgumentException("Resource must be both parent and child");
}
this.id = id;
this.parent = parent;
this.resource = resource;
this.children = new HashSet<>();
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ public Stream<Resource> traverse() {
return Stream.of(this.get());
}

// Collection
/* Implementations for Collection interface */

@Override
public int size() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ public Resource get() {
return resource;
}

// Collection
/* Implementations for Collection interface */

@Override
public int size() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,11 +57,13 @@ void testTreeBuildsCorrectly() {
tree.add(file1);

assertThat(tree.stream().collect(Collectors.toSet()))
.as("Tree stream should contain all (three) elements added to the tree")
.isEqualTo(Set.of(root, some, file1));

tree.add(file2);

assertThat(tree.stream().collect(Collectors.toSet()))
.as("Tree stream should contain all (four) elements added to the tree")
.isEqualTo(Set.of(root, some, file1, file2));
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import uk.gov.gchq.palisade.Generated;
import uk.gov.gchq.palisade.client.java.util.Util;

import java.net.URI;
Expand Down Expand Up @@ -51,7 +52,7 @@ private interface StringProperty<T> {
private static final String TOKEN_PARAM = "%25t"; // % is percent-encoded to %25
private static final int KV_PAIR_LENGTH = 2;

// Configurable
// Keys for configurable properties
/**
* The userId for connecting to the palisade services, will be used for all requests.
* <p>
Expand Down Expand Up @@ -80,15 +81,15 @@ private interface StringProperty<T> {
*/
public static final String POLL_SECONDS = "poll";

// Allowed user-configurable properties
// Allowed user-configurable properties and readers for them (from String to T)
protected static final Map<String, StringProperty<?>> WHITELIST_PROPERTIES = Map.of(
USER_ID, String::new,
SSL_ENABLED, Boolean::valueOf,
HTTP2_ENABLED, Boolean::valueOf,
POLL_SECONDS, Long::valueOf
);

// Dafaults
// Static keys which are not permitted to be configurable
/**
* Relative path from cluster root to palisade-service request endpoint.
* <p>
Expand All @@ -107,7 +108,14 @@ private interface StringProperty<T> {
* Static, default '/read/chunked'
*/
public static final String DATA_PATH = "data.path";
/**
* Map from service-names to URIs
* <p>
* Static, default empty map
*/
public static final String DATA_SERVICE_MAP = "data.service-map";

// Defaults for above static and configurable keys
protected static final Map<String, Object> DEFAULT_PROPERTIES = Map.of(
// Static
PALISADE_PATH, "/palisade/api/registerDataRequest",
Expand All @@ -119,7 +127,7 @@ private interface StringProperty<T> {
POLL_SECONDS, 3600L
);

// Required and derived
// Required and derived keys for connection properties
/**
* The original URI Spec string passed to the configuration class.
* <p>
Expand All @@ -139,6 +147,7 @@ private interface StringProperty<T> {
*/
public static final String FILTERED_RESOURCE_URI = "filtered-resource.uri";


private final Map<String, Object> properties = new TreeMap<>();

protected Configuration(final Map<String, Object> properties) {
Expand Down Expand Up @@ -195,7 +204,7 @@ public static Configuration create(final URI spec) {
var palisadeScheme = "http";
var filteredResourceScheme = "ws";
var dataScheme = "http";
// Determine SSL config
// Determine SSL config for URI schemes
if (config.<Boolean>get(SSL_ENABLED).equals(Boolean.TRUE)) {
palisadeScheme = "https";
filteredResourceScheme = "wss";
Expand All @@ -214,6 +223,7 @@ public static Configuration create(final URI spec) {
DATA_SERVICE_MAP, Map.of("data-service", defaultDataUri)
));

// Log properties map for debugging
LOGGER.debug("Using spec {} built config:", spec);
config.properties.forEach((key, value) -> LOGGER.debug("{} = {}", key, value));

Expand Down Expand Up @@ -255,6 +265,7 @@ private static Map<String, Object> parseQueryParams(final String queryParamStrin
}

@Override
@Generated
public boolean equals(final Object o) {
if (this == o) {
return true;
Expand All @@ -267,11 +278,13 @@ public boolean equals(final Object o) {
}

@Override
@Generated
public int hashCode() {
return Objects.hash(properties);
}

@Override
@Generated
public String toString() {
return properties.toString();
}
Expand Down
Loading