Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NIFI-13632: Updating content access. #9221

Merged
merged 10 commits into from
Sep 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/ci-workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ env:
-pl -:nifi-ui
-pl -:nifi-jolt-nar
-pl -:nifi-jolt-transform-json-ui
-pl -:nifi-standard-content-viewer-nar
-pl -:nifi-standard-content-viewer
MAVEN_VERIFY_COMMAND: >-
verify
--show-version
Expand Down
3 changes: 3 additions & 0 deletions nifi-assembly/LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -2869,4 +2869,7 @@ For the binary distribution of nifi-ui see its 3rdpartylicenses.txt
for additional license and notice information.

For the binary distribution of nifi-jolt-transform-json-ui see its 3rdpartylicenses.txt
for additional license and notice information.

For the binary distribution of nifi-standard-content-viewer see its 3rdpartylicenses.txt
for additional license and notice information.
3 changes: 3 additions & 0 deletions nifi-assembly/NOTICE
Original file line number Diff line number Diff line change
Expand Up @@ -2577,4 +2577,7 @@ For the binary distribution of nifi-ui see its 3rdpartylicenses.txt
for additional license and notice information.

For the binary distribution of nifi-jolt-transform-json-ui see its 3rdpartylicenses.txt
for additional license and notice information.

For the binary distribution of nifi-standard-content-viewer see its 3rdpartylicenses.txt
for additional license and notice information.
12 changes: 6 additions & 6 deletions nifi-assembly/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -280,6 +280,12 @@ language governing permissions and limitations under the License. -->
<version>2.0.0-SNAPSHOT</version>
<type>nar</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-standard-content-viewer-nar</artifactId>
<version>2.0.0-SNAPSHOT</version>
<type>nar</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-jetty-nar</artifactId>
Expand Down Expand Up @@ -814,12 +820,6 @@ language governing permissions and limitations under the License. -->
<version>2.0.0-SNAPSHOT</version>
<type>nar</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-image-viewer-nar</artifactId>
<version>2.0.0-SNAPSHOT</version>
<type>nar</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-compress-nar</artifactId>
Expand Down
20 changes: 1 addition & 19 deletions nifi-code-coverage/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -604,15 +604,9 @@
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-web-content-access</artifactId>
<artifactId>nifi-content-viewer-utils</artifactId>
<version>2.0.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-web-content-viewer</artifactId>
<version>2.0.0-SNAPSHOT</version>
<type>war</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-web-optimistic-locking</artifactId>
Expand Down Expand Up @@ -1087,12 +1081,6 @@
<artifactId>nifi-ldap-iaa-providers</artifactId>
<version>2.0.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-image-viewer</artifactId>
<version>2.0.0-SNAPSHOT</version>
<type>war</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-media-processors</artifactId>
Expand Down Expand Up @@ -1288,12 +1276,6 @@
<artifactId>nifi-sql-reporting-tasks</artifactId>
<version>2.0.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-standard-content-viewer</artifactId>
<version>2.0.0-SNAPSHOT</version>
<type>war</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-standard-parameter-providers</artifactId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,27 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-web</artifactId>
<artifactId>nifi-commons</artifactId>
<version>2.0.0-SNAPSHOT</version>
</parent>
<artifactId>nifi-web-content-access</artifactId>
<artifactId>nifi-content-viewer-utils</artifactId>
<dependencies>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-framework-api</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
</dependency>
<dependency>
<groupId>jakarta.servlet</groupId>
<artifactId>jakarta.servlet-api</artifactId>
<version>${servlet-api.version}</version>
</dependency>
<dependency>
<groupId>jakarta.ws.rs</groupId>
<artifactId>jakarta.ws.rs-api</artifactId>
</dependency>
</dependencies>
</project>
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.web;

import jakarta.servlet.http.HttpServletRequest;
import jakarta.ws.rs.core.UriBuilder;
import org.apache.commons.lang3.StringUtils;

import java.net.URI;

/**
* An implementation of the ConfigurationRequestContext that retrieves configuration
* from a HttpServletRequest instance.
*/
public class HttpServletContentRequestContext implements ContentRequestContext {

private final String dataRef;
private final String clusterNodeId;
private final String clientId;

private static final String PROXY_CONTEXT_PATH_HTTP_HEADER = "X-ProxyContextPath";
private static final String FORWARDED_CONTEXT_HTTP_HEADER = "X-Forwarded-Context";
private static final String FORWARDED_PREFIX_HTTP_HEADER = "X-Forwarded-Prefix";

public HttpServletContentRequestContext(HttpServletRequest request) {
final String ref = request.getParameter("ref");
final UriBuilder refUriBuilder = UriBuilder.fromUri(ref);

// base the data ref on the request parameter but ensure the scheme is based off the incoming request...
// this is necessary for scenario's where the NiFi instance is behind a proxy running a different scheme
refUriBuilder.scheme(request.getScheme());

// If there is path context from a proxy, remove it since this request will be used inside the cluster
final String proxyContextPath = getFirstHeaderValue(request, PROXY_CONTEXT_PATH_HTTP_HEADER, FORWARDED_CONTEXT_HTTP_HEADER, FORWARDED_PREFIX_HTTP_HEADER);
if (StringUtils.isNotBlank(proxyContextPath)) {
refUriBuilder.replacePath(StringUtils.substringAfter(UriBuilder.fromUri(ref).build().getPath(), proxyContextPath));
}

final URI refUri = refUriBuilder.build();

final String query = refUri.getQuery();

String rawClusterNodeId = null;
if (query != null) {
final String[] queryParameters = query.split("&");

for (int i = 0; i < queryParameters.length; i++) {
if (queryParameters[0].startsWith("clusterNodeId=")) {
rawClusterNodeId = StringUtils.substringAfterLast(queryParameters[0], "clusterNodeId=");
}
}
}
final String clusterNodeId = rawClusterNodeId;

this.dataRef = refUri.toString();
this.clusterNodeId = clusterNodeId;
this.clientId = request.getParameter("clientId");
}

/**
* Returns the value for the first key discovered when inspecting the current request. Will
* return null if there are no keys specified or if none of the specified keys are found.
*
* @param keys http header keys
* @return the value for the first key found
*/
private String getFirstHeaderValue(HttpServletRequest httpServletRequest, final String... keys) {
if (keys == null) {
return null;
}

for (final String key : keys) {
final String value = httpServletRequest.getHeader(key);

// if we found an entry for this key, return the value
if (value != null) {
return value;
}
}

// unable to find any matching keys
return null;
}

@Override
public String getDataUri() {
return this.dataRef;
}

@Override
public String getClusterNodeId() {
return this.clusterNodeId;
}

@Override
public String getClientId() {
return this.clientId;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,19 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-web</artifactId>
<artifactId>nifi-commons</artifactId>
<version>2.0.0-SNAPSHOT</version>
</parent>
<artifactId>nifi-custom-ui-utilities</artifactId>
<artifactId>nifi-custom-ui-utils</artifactId>
<dependencies>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-framework-api</artifactId>
</dependency>
<dependency>
<groupId>jakarta.servlet</groupId>
<artifactId>jakarta.servlet-api</artifactId>
<version>${servlet-api.version}</version>
</dependency>
</dependencies>
</project>
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,6 @@ public class NiFiProperties extends ApplicationProperties {
public static final String CONTENT_ARCHIVE_BACK_PRESSURE_PERCENTAGE = "nifi.content.repository.archive.backpressure.percentage";
public static final String CONTENT_ARCHIVE_ENABLED = "nifi.content.repository.archive.enabled";
public static final String CONTENT_ARCHIVE_CLEANUP_FREQUENCY = "nifi.content.repository.archive.cleanup.frequency";
public static final String CONTENT_VIEWER_URL = "nifi.content.viewer.url";

// flowfile repository properties
public static final String FLOWFILE_REPOSITORY_IMPLEMENTATION = "nifi.flowfile.repository.implementation";
Expand Down
2 changes: 2 additions & 0 deletions nifi-commons/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
<packaging>pom</packaging>
<modules>
<module>nifi-build</module>
<module>nifi-custom-ui-utils</module>
<module>nifi-data-provenance-utils</module>
<module>nifi-deprecation-log</module>
<module>nifi-expression-language</module>
Expand Down Expand Up @@ -62,6 +63,7 @@
<module>nifi-hashicorp-vault-api</module>
<module>nifi-web-client</module>
<module>nifi-web-client-api</module>
<module>nifi-content-viewer-utils</module>
<module>nifi-web-servlet-shared</module>
<module>nifi-web-utils</module>
<module>nifi-write-ahead-log</module>
Expand Down
1 change: 0 additions & 1 deletion nifi-docs/src/main/asciidoc/administration-guide.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -3160,7 +3160,6 @@ When not set, the default value is derived as 2% greater than `nifi.content.repo
For example, if `nifi.content.repository.archive.max.usage.percentage` is `50%` and `nifi.content.repository.archive.backpressure.percentage` is not set, the effective value of `nifi.content.repository.archive.backpressure.percentage` will be `52%`.
|`nifi.content.repository.archive.enabled`|To enable content archiving, set this to `true` and specify a value for the `nifi.content.repository.archive.max.usage.percentage` property above. Content archiving enables the provenance UI to view or replay content that is no longer in a dataflow queue. By default, archiving is enabled.
|`nifi.content.repository.always.sync`|If set to `true`, any change to the repository will be synchronized to the disk, meaning that NiFi will ask the operating system not to cache the information. This is very expensive and can significantly reduce NiFi performance. However, if it is `false`, there could be the potential for data loss if either there is a sudden power loss or the operating system crashes. The default value is `false`.
|`nifi.content.viewer.url`|The URL for a web-based content viewer if one is available. The default is `../nifi-content-viewer/`.
|`nifi.content.repository.archive.cleanup.frequency`| The frequency with which to schedule the content archive clean up task. The default value is `1 Second`. A value lower than `1 Second` is not allowed.
|====

Expand Down
4 changes: 1 addition & 3 deletions nifi-docs/src/main/asciidoc/user-guide.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2408,9 +2408,7 @@ a Connection's context menu. The listing will return the top 100 FlowFiles in th
configured priority. The listing can be performed even if the source and destination are actively running.

Additionally, details for a FlowFile in the listing can be viewed by clicking the "Details" button (image:iconDetails.png["Details"]) in the left most column. From here, the FlowFile details and attributes are available as well as buttons for
downloading or viewing the content. Viewing the content is only available if the `nifi.content.viewer.url` has been configured.
If the source or destination of the Connection are actively running, there is a chance that the desired FlowFile will
no longer be in the active queue.
downloading or viewing the content. If the source or destination of the Connection are actively running, there is a chance that the desired FlowFile will no longer be in the active queue.

The FlowFiles enqueued in a Connection can also be deleted when necessary. The removal of the FlowFiles is initiated
via `Empty queue` in the Connection's context menu. This action can also be performed if the source and destination
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ language governing permissions and limitations under the License. -->
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-custom-ui-utilities</artifactId>
<artifactId>nifi-custom-ui-utils</artifactId>
<version>2.0.0-SNAPSHOT</version>
</dependency>
<dependency>
Expand Down Expand Up @@ -89,7 +89,7 @@ language governing permissions and limitations under the License. -->
<build>
<plugins>
<!--
Unpack the built nifi front end application
Unpack the built jolt front end application
-->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
Expand All @@ -113,7 +113,7 @@ language governing permissions and limitations under the License. -->
</executions>
</plugin>
<!--
Include the NiFi UI and exclude the JAR dependency
Include the Jolt UI and exclude the JAR dependency
-->
<plugin>
<artifactId>maven-war-plugin</artifactId>
Expand Down
Loading