Skip to content

Commit

Permalink
NIFI-13632: Updating content access. (#9221)
Browse files Browse the repository at this point in the history
* NIFI-13632: Updating content access.
- Removing existing content viewer application.
- Introduced new front end based content viewer applicaiton.
- Introduced new bundled content viewers for hex and images.
- Deleted previous image content viewer.
- Migrated existing standard content viewer (which handles json, xml, yaml, csv, and text) to utilize new content access interface.
- Moved standard content viewer into its own NAR.
- Moved and renamed custom ui utils and content viewer utils into nifi-common.
- Added mime type to FlowFileSummary response payload to help drive the initially opened content viewer.

* NIFI-13632: Fixing rat issue.

* NIFI-13632: Fixing CI issue.

* NIFI-13632: Removing standard content viewer war from code coverage.

* NIFI-13632: Addressing review feedback.

* NIFI-13632: Fixing import.

* NIFI-13632: Further simplification of content viewer state.

* NIFI-13632: Removing unneeded code.

* NIFI-13632: Adding a min height around codemirror.

* NIFI-13632: Addressing review feedback.
- Rendering mime type in the content viewer.
- Preventing additions to the browser history as the user changes how the content is viewed.
- Appending forward slash to content viewer UIs to prevent unnecessary redirect.
- Removing Standard Content Viewer title based on review feedback.
- Fixing double scroll bar in the Standard Content Viewer on small browser heights.
- Removed unnecessary application.yml.
- Bumping spring boot version.

This closes #9221
  • Loading branch information
mcgilman authored Sep 6, 2024
1 parent 85ca96c commit 39ab312
Show file tree
Hide file tree
Showing 197 changed files with 4,836 additions and 6,442 deletions.
2 changes: 2 additions & 0 deletions .github/workflows/ci-workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ env:
-pl -:nifi-ui
-pl -:nifi-jolt-nar
-pl -:nifi-jolt-transform-json-ui
-pl -:nifi-standard-content-viewer-nar
-pl -:nifi-standard-content-viewer
MAVEN_VERIFY_COMMAND: >-
verify
--show-version
Expand Down
3 changes: 3 additions & 0 deletions nifi-assembly/LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -2869,4 +2869,7 @@ For the binary distribution of nifi-ui see its 3rdpartylicenses.txt
for additional license and notice information.

For the binary distribution of nifi-jolt-transform-json-ui see its 3rdpartylicenses.txt
for additional license and notice information.

For the binary distribution of nifi-standard-content-viewer see its 3rdpartylicenses.txt
for additional license and notice information.
3 changes: 3 additions & 0 deletions nifi-assembly/NOTICE
Original file line number Diff line number Diff line change
Expand Up @@ -2577,4 +2577,7 @@ For the binary distribution of nifi-ui see its 3rdpartylicenses.txt
for additional license and notice information.

For the binary distribution of nifi-jolt-transform-json-ui see its 3rdpartylicenses.txt
for additional license and notice information.

For the binary distribution of nifi-standard-content-viewer see its 3rdpartylicenses.txt
for additional license and notice information.
12 changes: 6 additions & 6 deletions nifi-assembly/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -280,6 +280,12 @@ language governing permissions and limitations under the License. -->
<version>2.0.0-SNAPSHOT</version>
<type>nar</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-standard-content-viewer-nar</artifactId>
<version>2.0.0-SNAPSHOT</version>
<type>nar</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-jetty-nar</artifactId>
Expand Down Expand Up @@ -814,12 +820,6 @@ language governing permissions and limitations under the License. -->
<version>2.0.0-SNAPSHOT</version>
<type>nar</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-image-viewer-nar</artifactId>
<version>2.0.0-SNAPSHOT</version>
<type>nar</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-compress-nar</artifactId>
Expand Down
20 changes: 1 addition & 19 deletions nifi-code-coverage/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -604,15 +604,9 @@
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-web-content-access</artifactId>
<artifactId>nifi-content-viewer-utils</artifactId>
<version>2.0.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-web-content-viewer</artifactId>
<version>2.0.0-SNAPSHOT</version>
<type>war</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-web-optimistic-locking</artifactId>
Expand Down Expand Up @@ -1087,12 +1081,6 @@
<artifactId>nifi-ldap-iaa-providers</artifactId>
<version>2.0.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-image-viewer</artifactId>
<version>2.0.0-SNAPSHOT</version>
<type>war</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-media-processors</artifactId>
Expand Down Expand Up @@ -1288,12 +1276,6 @@
<artifactId>nifi-sql-reporting-tasks</artifactId>
<version>2.0.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-standard-content-viewer</artifactId>
<version>2.0.0-SNAPSHOT</version>
<type>war</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-standard-parameter-providers</artifactId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,27 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-web</artifactId>
<artifactId>nifi-commons</artifactId>
<version>2.0.0-SNAPSHOT</version>
</parent>
<artifactId>nifi-web-content-access</artifactId>
<artifactId>nifi-content-viewer-utils</artifactId>
<dependencies>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-framework-api</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
</dependency>
<dependency>
<groupId>jakarta.servlet</groupId>
<artifactId>jakarta.servlet-api</artifactId>
<version>${servlet-api.version}</version>
</dependency>
<dependency>
<groupId>jakarta.ws.rs</groupId>
<artifactId>jakarta.ws.rs-api</artifactId>
</dependency>
</dependencies>
</project>
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.web;

import jakarta.servlet.http.HttpServletRequest;
import jakarta.ws.rs.core.UriBuilder;
import org.apache.commons.lang3.StringUtils;

import java.net.URI;

/**
* An implementation of the ConfigurationRequestContext that retrieves configuration
* from a HttpServletRequest instance.
*/
public class HttpServletContentRequestContext implements ContentRequestContext {

private final String dataRef;
private final String clusterNodeId;
private final String clientId;

private static final String PROXY_CONTEXT_PATH_HTTP_HEADER = "X-ProxyContextPath";
private static final String FORWARDED_CONTEXT_HTTP_HEADER = "X-Forwarded-Context";
private static final String FORWARDED_PREFIX_HTTP_HEADER = "X-Forwarded-Prefix";

public HttpServletContentRequestContext(HttpServletRequest request) {
final String ref = request.getParameter("ref");
final UriBuilder refUriBuilder = UriBuilder.fromUri(ref);

// base the data ref on the request parameter but ensure the scheme is based off the incoming request...
// this is necessary for scenario's where the NiFi instance is behind a proxy running a different scheme
refUriBuilder.scheme(request.getScheme());

// If there is path context from a proxy, remove it since this request will be used inside the cluster
final String proxyContextPath = getFirstHeaderValue(request, PROXY_CONTEXT_PATH_HTTP_HEADER, FORWARDED_CONTEXT_HTTP_HEADER, FORWARDED_PREFIX_HTTP_HEADER);
if (StringUtils.isNotBlank(proxyContextPath)) {
refUriBuilder.replacePath(StringUtils.substringAfter(UriBuilder.fromUri(ref).build().getPath(), proxyContextPath));
}

final URI refUri = refUriBuilder.build();

final String query = refUri.getQuery();

String rawClusterNodeId = null;
if (query != null) {
final String[] queryParameters = query.split("&");

for (int i = 0; i < queryParameters.length; i++) {
if (queryParameters[0].startsWith("clusterNodeId=")) {
rawClusterNodeId = StringUtils.substringAfterLast(queryParameters[0], "clusterNodeId=");
}
}
}
final String clusterNodeId = rawClusterNodeId;

this.dataRef = refUri.toString();
this.clusterNodeId = clusterNodeId;
this.clientId = request.getParameter("clientId");
}

/**
* Returns the value for the first key discovered when inspecting the current request. Will
* return null if there are no keys specified or if none of the specified keys are found.
*
* @param keys http header keys
* @return the value for the first key found
*/
private String getFirstHeaderValue(HttpServletRequest httpServletRequest, final String... keys) {
if (keys == null) {
return null;
}

for (final String key : keys) {
final String value = httpServletRequest.getHeader(key);

// if we found an entry for this key, return the value
if (value != null) {
return value;
}
}

// unable to find any matching keys
return null;
}

@Override
public String getDataUri() {
return this.dataRef;
}

@Override
public String getClusterNodeId() {
return this.clusterNodeId;
}

@Override
public String getClientId() {
return this.clientId;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,19 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-web</artifactId>
<artifactId>nifi-commons</artifactId>
<version>2.0.0-SNAPSHOT</version>
</parent>
<artifactId>nifi-custom-ui-utilities</artifactId>
<artifactId>nifi-custom-ui-utils</artifactId>
<dependencies>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-framework-api</artifactId>
</dependency>
<dependency>
<groupId>jakarta.servlet</groupId>
<artifactId>jakarta.servlet-api</artifactId>
<version>${servlet-api.version}</version>
</dependency>
</dependencies>
</project>
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,6 @@ public class NiFiProperties extends ApplicationProperties {
public static final String CONTENT_ARCHIVE_BACK_PRESSURE_PERCENTAGE = "nifi.content.repository.archive.backpressure.percentage";
public static final String CONTENT_ARCHIVE_ENABLED = "nifi.content.repository.archive.enabled";
public static final String CONTENT_ARCHIVE_CLEANUP_FREQUENCY = "nifi.content.repository.archive.cleanup.frequency";
public static final String CONTENT_VIEWER_URL = "nifi.content.viewer.url";

// flowfile repository properties
public static final String FLOWFILE_REPOSITORY_IMPLEMENTATION = "nifi.flowfile.repository.implementation";
Expand Down
2 changes: 2 additions & 0 deletions nifi-commons/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
<packaging>pom</packaging>
<modules>
<module>nifi-build</module>
<module>nifi-custom-ui-utils</module>
<module>nifi-data-provenance-utils</module>
<module>nifi-deprecation-log</module>
<module>nifi-expression-language</module>
Expand Down Expand Up @@ -62,6 +63,7 @@
<module>nifi-hashicorp-vault-api</module>
<module>nifi-web-client</module>
<module>nifi-web-client-api</module>
<module>nifi-content-viewer-utils</module>
<module>nifi-web-servlet-shared</module>
<module>nifi-web-utils</module>
<module>nifi-write-ahead-log</module>
Expand Down
1 change: 0 additions & 1 deletion nifi-docs/src/main/asciidoc/administration-guide.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -3160,7 +3160,6 @@ When not set, the default value is derived as 2% greater than `nifi.content.repo
For example, if `nifi.content.repository.archive.max.usage.percentage` is `50%` and `nifi.content.repository.archive.backpressure.percentage` is not set, the effective value of `nifi.content.repository.archive.backpressure.percentage` will be `52%`.
|`nifi.content.repository.archive.enabled`|To enable content archiving, set this to `true` and specify a value for the `nifi.content.repository.archive.max.usage.percentage` property above. Content archiving enables the provenance UI to view or replay content that is no longer in a dataflow queue. By default, archiving is enabled.
|`nifi.content.repository.always.sync`|If set to `true`, any change to the repository will be synchronized to the disk, meaning that NiFi will ask the operating system not to cache the information. This is very expensive and can significantly reduce NiFi performance. However, if it is `false`, there could be the potential for data loss if either there is a sudden power loss or the operating system crashes. The default value is `false`.
|`nifi.content.viewer.url`|The URL for a web-based content viewer if one is available. The default is `../nifi-content-viewer/`.
|`nifi.content.repository.archive.cleanup.frequency`| The frequency with which to schedule the content archive clean up task. The default value is `1 Second`. A value lower than `1 Second` is not allowed.
|====

Expand Down
4 changes: 1 addition & 3 deletions nifi-docs/src/main/asciidoc/user-guide.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2408,9 +2408,7 @@ a Connection's context menu. The listing will return the top 100 FlowFiles in th
configured priority. The listing can be performed even if the source and destination are actively running.

Additionally, details for a FlowFile in the listing can be viewed by clicking the "Details" button (image:iconDetails.png["Details"]) in the left most column. From here, the FlowFile details and attributes are available as well as buttons for
downloading or viewing the content. Viewing the content is only available if the `nifi.content.viewer.url` has been configured.
If the source or destination of the Connection are actively running, there is a chance that the desired FlowFile will
no longer be in the active queue.
downloading or viewing the content. If the source or destination of the Connection are actively running, there is a chance that the desired FlowFile will no longer be in the active queue.

The FlowFiles enqueued in a Connection can also be deleted when necessary. The removal of the FlowFiles is initiated
via `Empty queue` in the Connection's context menu. This action can also be performed if the source and destination
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ language governing permissions and limitations under the License. -->
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-custom-ui-utilities</artifactId>
<artifactId>nifi-custom-ui-utils</artifactId>
<version>2.0.0-SNAPSHOT</version>
</dependency>
<dependency>
Expand Down Expand Up @@ -89,7 +89,7 @@ language governing permissions and limitations under the License. -->
<build>
<plugins>
<!--
Unpack the built nifi front end application
Unpack the built jolt front end application
-->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
Expand All @@ -113,7 +113,7 @@ language governing permissions and limitations under the License. -->
</executions>
</plugin>
<!--
Include the NiFi UI and exclude the JAR dependency
Include the Jolt UI and exclude the JAR dependency
-->
<plugin>
<artifactId>maven-war-plugin</artifactId>
Expand Down
Loading

0 comments on commit 39ab312

Please sign in to comment.