From fea010ae4c9c1d00299c73b3bb51657a66376c64 Mon Sep 17 00:00:00 2001 From: "Kevin W. Wall" Date: Wed, 12 Apr 2023 18:34:10 -0400 Subject: [PATCH] Initial 2.5.2.0 release preparation (#784) * Updates to ESAPI HTTPUtilities.getFileUploads methods to address CVE-2023-24998. * Updates to AntiSamy 1.7.3 to address CVE-2023-26119. * Fixed package level Javadoc for org.owasp.esapi package. * Update example scripts so they work for recent ESAPI releases. * Mention signed commits now required in CONTRIBUTING-TO-ESAPI.txt. * Updated plugins and dependencies to latest versions that work. * Updated date.prev_release to correctly compute CHANGELOG for 'mvn site'. * Update boilerplate in template file used for creating new release notes. * Created scripts/varrs.2.6.2.0, used to generate 2.5.2.0 release notes. * Add missing class level Javadoc to org.owasp.esapi.ValidationRule. * Created new 2.5.2.0 release notes. * Change reference of latest release from 2.5.1.0 to 2.5.2.0 in README.md. * Make ANONYMOUS user 'final' in org.owasp.esapi.User interface. * Added 2 new properties (HttpUtilities.MaxUploadFileCount & HttpUtilities.FileUploadAllowAnonymousUser) to ESAPI.properties files to address CVE-2023-24998. * Add 2 new property names to PropNames class that are used to address CVE-2023-24998: HttpUtilities.MaxUploadFileCount and HttpUtilities.FileUploadAllowAnonymousUser. * Changed start-up log message for ESAPI WAF to make it specific to the _ESAPI_ WAF. * Extensive Javadoc updates, especially to the HTTPUtilities.getFileUploads methods. * Changes to DefaultHTTPUtilities to address CVE-2023-24998. * Add new tests for getFileUploads method to test CVE-2023-24998 remediation (as much as possible). * Commented out new JUnit test, HTTPUtilitiesTest.testGetFileUploadsUnauthenticatedUser for reasons noted in code. --- CONTRIBUTING-TO-ESAPI.txt | 9 + README.md | 4 +- configuration/esapi/ESAPI.properties | 25 +++ .../esapi4java-core-2.5.2.0-release-notes.txt | 188 ++++++++++++++++++ pom.xml | 26 +-- ...esapi4java-core-TEMPLATE-release-notes.txt | 16 +- scripts/vars.2.5.2.0 | 14 ++ src/examples/README | 12 +- src/examples/scripts/compile.sh | 10 +- src/examples/scripts/encryptProperties.sh | 13 +- src/examples/scripts/persistEncryptedData.sh | 14 +- src/examples/scripts/runClass.sh | 13 +- src/examples/scripts/setMasterKey.sh | 24 ++- src/examples/scripts/setenv-git.sh | 40 ++++ src/examples/scripts/setenv-svn.sh | 4 +- src/examples/scripts/setenv-zip.sh | 11 +- .../java/org/owasp/esapi/HTTPUtilities.java | 103 +++++++++- src/main/java/org/owasp/esapi/PropNames.java | 2 + src/main/java/org/owasp/esapi/User.java | 2 +- .../java/org/owasp/esapi/ValidationRule.java | 5 + src/main/java/org/owasp/esapi/package.html | 4 +- .../esapi/reference/DefaultHTTPUtilities.java | 86 +++++++- .../ESAPIWebApplicationFirewallFilter.java | 2 +- .../esapi/reference/HTTPUtilitiesTest.java | 185 ++++++++++++++++- src/test/resources/esapi/ESAPI.properties | 25 +++ 25 files changed, 755 insertions(+), 82 deletions(-) create mode 100644 documentation/esapi4java-core-2.5.2.0-release-notes.txt create mode 100644 scripts/vars.2.5.2.0 create mode 100755 src/examples/scripts/setenv-git.sh diff --git a/CONTRIBUTING-TO-ESAPI.txt b/CONTRIBUTING-TO-ESAPI.txt index f97e9c00b..970efcac1 100644 --- a/CONTRIBUTING-TO-ESAPI.txt +++ b/CONTRIBUTING-TO-ESAPI.txt @@ -12,6 +12,15 @@ A Special Note on GitHub Authentication: Please see https://github.blog/2020-12-15-token-authentication-requirements-for-git-operations/ for details and plan accordingly. +A Special Note Regarding Making Commits for PRs + Shortly after the 2.5.1.0 ESAPI release in late November 2022, the ESAPI + team decided to lock down the 'develop' amd 'main' branches. Merges from + PRs are done to the 'develop' branch. That means that if you intend to + contribute to ESAPI, you must be signing your commits. Please see the + GitHub instructions at + https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits + for details. + Finding Something Interesting to Work on: diff --git a/README.md b/README.md index 6ca18e3ae..707624be2 100644 --- a/README.md +++ b/README.md @@ -45,7 +45,7 @@ Development for the "next generation" of ESAPI (starting with ESAPI 3.0), will b GitHub repository at [https://github.com/ESAPI/esapi-java](https://github.com/ESAPI/esapi-java). **IMPORTANT NOTES:** -* The default branch for ESAPI legacy is the 'develop' branch (rather than the 'main' (formerly 'master') branch), where future development, bug fixes, etc. are now being done. The 'main' branch is now marked as "protected"; it reflects the latest stable ESAPI release (2.5.1.0 as of this date). Note that this change of making the 'develop' branch the default may affect any pull requests that you were intending to make. +* The default branch for ESAPI legacy is the 'develop' branch (rather than the 'main' (formerly 'master') branch), where future development, bug fixes, etc. are now being done. The 'main' branch is now marked as "protected"; it reflects the latest stable ESAPI release (2.5.2.0 as of this date). Note that this change of making the 'develop' branch the default may affect any pull requests that you were intending to make. * Also, the *minimal* baseline Java version to use ESAPI is now Java 8. (This was changed from Java 7 during the 2.4.0.0 release.) * Support was dropped for Log4J 1 during ESAPI 2.5.0.0 release. If you need it, configure it via SLF4J. See the [2.5.0.0 release notes](https://github.com/ESAPI/esapi-java-legacy/blob/develop/documentation/esapi4java-core-2.5.0.0-release-notes.txt) @@ -79,7 +79,7 @@ link to the specific release notes. Starting with release 2.4.0.0, Java 8 or later is required. # Locating ESAPI Jar files -The [latest ESAPI release](https://github.com/ESAPI/esapi-java-legacy/releases/latest) is 2.5.1.0. +The [latest ESAPI release](https://github.com/ESAPI/esapi-java-legacy/releases/latest) is 2.5.2.0. All the *regular* ESAPI jars, with the exception of the ESAPI configuration jar (i.e., esapi-2.#.#.#-configuration.jar) and its associated detached GPG signature, are available from Maven Central. The ESAPI configuration diff --git a/configuration/esapi/ESAPI.properties b/configuration/esapi/ESAPI.properties index 19c34b729..030f61588 100644 --- a/configuration/esapi/ESAPI.properties +++ b/configuration/esapi/ESAPI.properties @@ -301,6 +301,7 @@ Encryptor.KDF.PRF=HmacSHA256 # headers, and CSRF tokens. # # Default file upload location (remember to escape backslashes with \\) +# HttpUtilities.UploadDir=C:\\ESAPI\\testUpload HttpUtilities.UploadTempDir=C:\\temp # Force flags on cookies, if you use HttpUtilities to set cookies @@ -335,6 +336,30 @@ HttpUtilities.httpQueryParamValueLength=500 # File upload configuration HttpUtilities.ApprovedUploadExtensions=.pdf,.doc,.docx,.ppt,.pptx,.xls,.xlsx,.rtf,.txt,.jpg,.png HttpUtilities.MaxUploadFileBytes=500000000 +# Maximum # of files that can be uploaded per HTTP request. +# Set to -1 for no maximum. Related to CVE-2023-24998. +HttpUtilities.MaxUploadFileCount=20 + +# Allowing anonymous users to do file uploads via HTTPUtilities.getFileUploads +# can make it easier for DoS attacks via uploading files easier. (See Security Bulletin #11, +# https://github.com/ESAPI/esapi-java-legacy/blob/develop/documentation/ESAPI-security-bulletin11.pdf +# for details). +# +# By default, we allow anonymous users to upload files because we can only rely on +# ESAPI.authenticator().getCurrentUser() to determine if a user associated +# with the current HTTP session is authenticated and almost no one uses the +# ESAPI Authenticator because the reference implementation is just a toy +# implementation and is not enterprise scalable. +# +# If you are using the ESAPI Authenticator (the ESAPI reference implementation +# or you've implemented your own custom one), then you can set this property value +# to 'false' to disallow anonymous (i.e., unauthenticated) users to upload +# files. However, if you are not using the ESAPI Authenticator, then you should +# probably leave this set to 'false', otherwise you will completely prevent the +# use of HTTPUtilities.getFileUploads methods. +# +HttpUtilities.FileUploadAllowAnonymousUser=true + # Using UTF-8 throughout your stack is highly recommended. That includes your database driver, # container, and any other technologies you may be using. Failure to do this may expose you # to Unicode transcoding injection attacks. Use of UTF-8 does not hinder internationalization. diff --git a/documentation/esapi4java-core-2.5.2.0-release-notes.txt b/documentation/esapi4java-core-2.5.2.0-release-notes.txt new file mode 100644 index 000000000..db4f48e0d --- /dev/null +++ b/documentation/esapi4java-core-2.5.2.0-release-notes.txt @@ -0,0 +1,188 @@ +Release notes for ESAPI 2.5.2.0 + Release date: 2023-04-12 + Project leaders: + -Kevin W. Wall + -Matt Seil + +Previous release: ESAPI 2.5.1.0, 2022-11-27 + + +Executive Summary: Important Things to Note for this Release +------------------------------------------------------------ +This is a patch release with the primary intent of updating some dependencies, one with a known DoS vulnerability and a more recent one with a potential RCE. From a vulnerability perspective, it addresses CVE-2023-24998 by upgrading to version 1.5 of Apache Commons File Uploads and adding the necessary call to FileBaseUpload.setFileCountMax(). It also updates to version 1.7.3 of AntiSamy to address CVE-2023-26119, a vulnerability in one of their dependencies. + +If you are not updating from the previous ESAPI release (2.5.1.0), you should go back and FIRST read all the subsequent release notes in turn. For instance, if you are currently on release 2.3.0.0 and upgrading to this release (2.5.2.0), you should MINIMALLY +read the sections "Changes Requiring Special Attention" in each of the subsequent release notes. So, going from release 2.3.0.0 to 2.5.2.0, you should in turn, read: + + esapi4java-core-2.4.0.0-release-notes.txt + esapi4java-core-2.5.0.0-release-notes.txt + esapi4java-core-2.5.1.0-release-notes.txt + esapi4java-core-2.5.2.0-release-notes.txt + +in that order. YOU HAVE BEEN WARNED!!! + +If your SCA tool is reporting any CVE from a direct or transitive dependency in ESAPI, before reporting it as an GitHub issue, please make sure that you review the vulnerability analysis written up in https://github.com/ESAPI/esapi-java-legacy/blob/develop/Vulnerability-Summary.md. Please email us or contact us in our GitHub Discussions page if you have questions about this. See also the SECURITY.md file to report any security issues with ESAPI. + + +================================================================================================================= + +Basic ESAPI facts +----------------- + +ESAPI 2.5.1.0 release: + 207 Java source files + 4292 JUnit tests in 131 Java source files (0 tests skipped) + +ESAPI 2.5.2.0 release: (unchanged since previous release) + 207 Java source files + 4293 JUnit tests in 131 Java source files (0 tests skipped, 1 commented out) + +7 GitHub Issues closed in this release, including those we've decided not to fix (marked 'wontfix' and 'falsepositive'). +(Reference: https://github.com/ESAPI/esapi-java-legacy/issues?q=is%3Aissue+state%3Aclosed+updated%3A%3E%3D2022-11-27) + +Issue # GitHub Issue Title +---------------------------------------------------------------------------------------------- +773 Esapi giving issue working with graal native image bug +770 latest version of ESAPI 2.5.1.0 not working with spring boot 3.0, it gives classNotFound for javax.servlet. duplicate enhancement +769 ESAPI 2.5.1.0 not working with spring boot 3.0, spring 6 bug +767 Add support for Jakarta Servlet API Specification enhancement [converted to Discussion #768] +764 unable to locate resource: esapi-java-logging.properties +761 JavaLogFactory is not loaded from ESAPI.properties file bug +760 Could not initialize class org. Owasp. Esapi. Reference. DefaultValidator bug + +----------------------------------------------------------------------------- + + Changes Requiring Special Attention + +----------------------------------------------------------------------------- + +Important JDK Support Announcement +* ESAPI 2.3.0.0 was the last Java release to support Java 7. ESAPI 2.4.0 requires using Java 8 or later. See the ESAPI 2.4.0.0 release notes (https://github.com/ESAPI/esapi-java-legacy/blob/develop/documentation/esapi4java-core-2.4.0.0-release-notes.txt) for details as to the reason. + - This means if your project requires Java 7, you must use ESAPI 2.3.0.0 or earlier. +* We are aware that ESAPI does not support Spring Boot 3.x or later or Spring Framework 6.x or later. + - This is because these projects use a version of Jakarta Servlet API that is incompatible with the the Java EE Servlet API. (The package names are different!) + - See Discussion #768 for more details. Please do NOT report this as an issue. + +Important ESAPI Logging Changes + +* Since ESAPI 2.5.0.0, support for logging directly via Log4J 1 has been removed. (This was two years after it haveing first been deprecated.) Thus, you only choice of ESAPI logging are + - java.util.logging (JUL), which as been the default since ESAPI 2.2.1.0. + * Set ESAPI.Logger=org.owasp.esapi.logging.java.JavaLogFactory in your ESAPI.properties file. + - SLF4J (which your choice of supported SLF4J logging implemmentation) + * Set ESAPI.Logger=org.owasp.esapi.logging.slf4j.Slf4JLogFactory in your ESAPI.properties file. +* Logger configuration notes - If you are migrating from prior to ESAPI 2.2.1.1, you will need to update your ESAPI.properties file as logging-related configuration as per the ESAPI 2.2.1.1 release notes, which may be found at: + https://github.com/ESAPI/esapi-java-legacy/blob/develop/documentation/esapi4java-core-2.2.1.1-release-notes.txt#L39-L78 + +If you use ESAPI 2.5.0.0 or later, you will get an ClassNotFoundException as the root cause if you still have your ESAPI.Logger property set to use Log4J because the org.owasp.esapi.logger.log4j.Log4JFactory class has been completely removed from the ESAPI jar. If you are dead set on continuing to use Log4J 1, you ought to be able to do so via SLF4J. The set up for Log4J 1 (which has not be tested), should be similar to configure ESAPI to use SLF4J with Log4J 2 as described here: + https://github.com/ESAPI/esapi-java-legacy/wiki/Using-ESAPI-with-SLF4J#slf4j-using-log4j-2x + +----------------------------------------------------------------------------- + + Remaining Known Issues / Problems + +----------------------------------------------------------------------------- +None known, other than the remaining open issues on GitHub. + +----------------------------------------------------------------------------- + + Other changes in this release, some of which not tracked via GitHub issues + +----------------------------------------------------------------------------- + +* Minor updates to README.md file with respect to version information. + +----------------------------------------------------------------------------- + +Developer Activity Report (Changes between release 2.5.1.0 and 2.5.2.0, i.e., between 2022-11-27 and 2023-04-12) +Generated manually based on merged PRs. All errors are the fault of kwwall and his inability to do simple arithmetic. + +Developer Total Total # of Unique # Merged +(GitHub ID) commits Files Changed PRs +======================================================== +davewichers 2 4 2 +josephWitthuhnTR 2 2 1 +dependabot 1 1 1 +kwwall 36 31 2 +======================================================== + Total merged PRs: 6 + +----------------------------------------------------------------------------- + +CHANGELOG: Create your own. May I suggest: + + git log --stat --since=2022-11-27 --reverse --pretty=medium + + which will show all the commits since just after the previous (2.5.1.0) release. + + Alternately, you can download the most recent ESAPI source and run + + mvn site + + which will create a CHANGELOG file named 'target/site/changelog.html' + + +----------------------------------------------------------------------------- + +Direct and Transitive Runtime and Test Dependencies: + + $ mvn -B dependency:tree + ... + [INFO] --- maven-dependency-plugin:3.5.0:tree (default-cli) @ esapi --- + [INFO] org.owasp.esapi:esapi:jar:2.5.2.0-SNAPSHOT + [INFO] +- javax.servlet:javax.servlet-api:jar:3.1.0:provided + [INFO] +- javax.servlet.jsp:javax.servlet.jsp-api:jar:2.3.3:provided + [INFO] +- xom:xom:jar:1.3.8:compile + [INFO] +- commons-beanutils:commons-beanutils:jar:1.9.4:compile + [INFO] | +- commons-logging:commons-logging:jar:1.2:compile + [INFO] | \- commons-collections:commons-collections:jar:3.2.2:compile + [INFO] +- commons-configuration:commons-configuration:jar:1.10:compile + [INFO] +- commons-lang:commons-lang:jar:2.6:compile + [INFO] +- commons-fileupload:commons-fileupload:jar:1.5:compile + [INFO] +- org.apache.commons:commons-collections4:jar:4.4:compile + [INFO] +- org.apache-extras.beanshell:bsh:jar:2.0b6:compile + [INFO] +- org.owasp.antisamy:antisamy:jar:1.7.2:compile + [INFO] | +- net.sourceforge.htmlunit:neko-htmlunit:jar:2.66.0:compile + [INFO] | +- org.apache.httpcomponents.client5:httpclient5:jar:5.2:compile + [INFO] | | \- org.apache.httpcomponents.core5:httpcore5-h2:jar:5.2:compile + [INFO] | +- org.apache.httpcomponents.core5:httpcore5:jar:5.2:compile + [INFO] | +- org.apache.xmlgraphics:batik-css:jar:1.16:compile + [INFO] | | +- org.apache.xmlgraphics:batik-shared-resources:jar:1.16:compile + [INFO] | | +- org.apache.xmlgraphics:batik-util:jar:1.16:compile + [INFO] | | | +- org.apache.xmlgraphics:batik-constants:jar:1.16:compile + [INFO] | | | \- org.apache.xmlgraphics:batik-i18n:jar:1.16:compile + [INFO] | | \- org.apache.xmlgraphics:xmlgraphics-commons:jar:2.7:compile + [INFO] | +- xerces:xercesImpl:jar:2.12.2:compile + [INFO] | \- xml-apis:xml-apis-ext:jar:1.3.04:compile + [INFO] +- org.slf4j:slf4j-api:jar:2.0.6:compile + [INFO] +- xml-apis:xml-apis:jar:1.4.01:compile + [INFO] +- commons-io:commons-io:jar:2.11.0:compile + [INFO] +- com.github.spotbugs:spotbugs-annotations:jar:4.7.3:compile + [INFO] | \- com.google.code.findbugs:jsr305:jar:3.0.2:compile + [INFO] +- commons-codec:commons-codec:jar:1.15:test + [INFO] +- junit:junit:jar:4.13.2:test + [INFO] +- org.bouncycastle:bcprov-jdk15on:jar:1.70:test + [INFO] +- org.hamcrest:hamcrest-core:jar:2.2:test + [INFO] | \- org.hamcrest:hamcrest:jar:2.2:test + [INFO] +- org.powermock:powermock-api-mockito2:jar:2.0.9:test + [INFO] | \- org.powermock:powermock-api-support:jar:2.0.9:test + [INFO] +- org.mockito:mockito-core:jar:3.12.4:test + [INFO] | +- net.bytebuddy:byte-buddy:jar:1.11.13:test + [INFO] | +- net.bytebuddy:byte-buddy-agent:jar:1.11.13:test + [INFO] | \- org.objenesis:objenesis:jar:3.2:test + [INFO] +- org.powermock:powermock-core:jar:2.0.9:test + [INFO] | \- org.javassist:javassist:jar:3.27.0-GA:test + [INFO] +- org.powermock:powermock-module-junit4:jar:2.0.9:test + [INFO] | \- org.powermock:powermock-module-junit4-common:jar:2.0.9:test + [INFO] +- org.powermock:powermock-reflect:jar:2.0.9:test + [INFO] \- org.openjdk.jmh:jmh-core:jar:1.36:test + [INFO] +- net.sf.jopt-simple:jopt-simple:jar:5.0.4:test + [INFO] \- org.apache.commons:commons-math3:jar:3.2:test + +----------------------------------------------------------------------------- + +Acknowledgments: + Thanks to my ESAPI co-contributors Matt Seil, Jeremiah Stacey, as well as all the ESAPI users who make our efforts worthwhile. Without you, there would be little point in maintaining this project. Lastly, a special shout-out to Joseph Witthuhn for submitting 2 PRs for this release. + +A special thanks to the ESAPI community from the ESAPI project co-leaders: + Kevin W. Wall (kwwall) <== The irresponsible party for these release notes! + Matt Seil (xeno6696) diff --git a/pom.xml b/pom.xml index e74a7ac5b..98ca83e0a 100644 --- a/pom.xml +++ b/pom.xml @@ -141,13 +141,13 @@ 2.0.0-M3 2.0.9 4.7.3 - 4.7.3.2 - 3.0.0-M9 + 4.7.3.4 + 3.0.0 1.8 - 2022-07-20 00:00:00 + 2022-11-27 00:00:00 @@ -246,7 +246,7 @@ org.owasp.antisamy antisamy - 1.7.2 + 1.7.3 @@ -421,7 +421,7 @@ org.apache.maven.plugins maven-release-plugin - 3.0.0-M7 + 3.0.0 org.codehaus.mojo @@ -439,7 +439,7 @@ org.cyclonedx cyclonedx-maven-plugin - 2.7.5 + 2.7.6 package @@ -521,7 +521,7 @@ org.apache.maven.plugins maven-deploy-plugin - 3.1.0 + 3.1.1 @@ -536,17 +536,17 @@ org.apache.maven.plugins maven-enforcer-plugin - 3.2.1 + 3.3.0 org.codehaus.mojo extra-enforcer-rules - 1.6.1 + 1.6.2 org.codehaus.mojo animal-sniffer-enforcer-rule - 1.22 + 1.23 @@ -623,7 +623,7 @@ org.apache.maven.plugins maven-install-plugin - 3.1.0 + 3.1.1 @@ -679,7 +679,7 @@ org.apache.maven.plugins maven-resources-plugin - 3.3.0 + 3.3.1 @@ -748,7 +748,7 @@ org.owasp dependency-check-maven - 8.1.2 + 8.2.1 1.0 ./suppressions.xml diff --git a/scripts/esapi4java-core-TEMPLATE-release-notes.txt b/scripts/esapi4java-core-TEMPLATE-release-notes.txt index 0decbe0e7..1d46a1cb6 100644 --- a/scripts/esapi4java-core-TEMPLATE-release-notes.txt +++ b/scripts/esapi4java-core-TEMPLATE-release-notes.txt @@ -16,10 +16,22 @@ Executive Summary: Important Things to Note for this Release @@@@ View previous release notes to see examples of what to put here. This is typical. YMMV. @@@@ Obviously, you should summarize any major changes / new features here. This is a patch release with the primary intent of updating some dependencies, some with known vulnerabilities. Details follow. +@@@@ Provide a sentence or to -For those of you using a Software Configuration Analysis (SCA) services such as Snyk, BlackDuck, Veracode SourceClear, OWASP Dependency Check, etc., you will notice that the 4 Log4J 1.x related CVEs are no longer flagged. This is because we have finally removed the Log4J 1.2.17 dependency in ESAPI 2.5.0.0. +Notes if you are not updating from the immediate previous release. release ${PREV_VERSION}: + * You need to read through the series of release notes FIRST, going in order. + * For example, if you were updating from an older ESAPI release (say, 2.3.0.0), you should go back and FIRST read all the subsequent release notes in turn. For instance, if you are currently on release 2.3.0.0 and upgrading to (say) release 2.x.y.z, you should MINIMALLY read the sections "Changes Requiring Special Attention" in each of the subsequent release notes. So, going from release 2.3.0.0 to 2.x.y.z, you should in turn, read: -Any remaining flagged vulnerabilities (e.g., CVE-2020-7791 for transitive dependency batik-i18n-1.14) are believed to be false postives. + esapi4java-core-2.4.0.0-release-notes.txt + esapi4java-core-2.5.0.0-release-notes.txt + esapi4java-core-2.5.1.0-release-notes.txt + esapi4java-core-2.5.2.0-release-notes.txt + ...etc., up through the current set of release notes... + esapi4java-core-2.x.y.z-release-notes.txt + +in that order. YOU HAVE BEEN WARNED!!! (These release notes are too large to put all this in a given document; very few read them thoroughly as it is.) + +If your SCA tool is reporting any CVE from a direct or transitive dependency in ESAPI, before reporting it as an GitHub issue, please make sure that you review the vulnerability analysis written up in https://github.com/ESAPI/esapi-java-legacy/blob/develop/Vulnerability-Summary.md. Please email us or contact us in our GitHub Discussions page if you have questions about this. See also the SECURITY.md file to report any security issues with ESAPI. You are encouraged to review the vulnerability analysis written up in https://github.com/ESAPI/esapi-java-legacy/blob/develop/Vulnerability-Summary.md and email us or contact us in our GitHub Discussions page if you have questions. diff --git a/scripts/vars.2.5.2.0 b/scripts/vars.2.5.2.0 new file mode 100644 index 000000000..77f49f324 --- /dev/null +++ b/scripts/vars.2.5.2.0 @@ -0,0 +1,14 @@ +# Do NOT edit this file directly. It will be created by the new createVarsFile.sh script, +# which should be run prior to the newReleaseNotes.sh script. + +# ESAPI (new / current) version +VERSION=2.5.2.0 + +# Previous ESAPI version +PREV_VERSION=2.5.1.0 + +# Release date of current version in yyyy-mm-dd format +YYYY_MM_DD_RELEASE_DATE=2023-03-31 + +# Previous ESAPI release date in same format +PREV_RELEASE_DATE=2022-11-27 diff --git a/src/examples/README b/src/examples/README index 9421f8819..126b6d85f 100644 --- a/src/examples/README +++ b/src/examples/README @@ -7,7 +7,7 @@ scripts - Some scripts to run that interact with the Java examples (but see NOTE: All scripts are meant to be run under *nix (or using something like - Cygwin under Windows) from under the 'scripts' directory using either + Cygwin or WSL under Windows) from under the 'scripts' directory using either bash or KornShell E.g., to compile the Java examples, execute: @@ -17,8 +17,8 @@ NOTE: All scripts are meant to be run under *nix (or using something like $ # "Dot" the appropriate environment file. Either $ # . ./setenv-zip.sh $ # if ESAPI downloaded via zip file or - $ # . ./setenv-svn.sh - $ # if ESAPI downloaded from SVN repository. For the + $ # . ./setenv-git.sh + $ # if ESAPI downloaded from a Git or Svn repository. For the $ # example of this README file, we will assume it was $ # downloaded via the zip file. $ . ./setenv-zip.sh @@ -29,9 +29,9 @@ NOTE: All scripts are meant to be run under *nix (or using something like Note that as delivered, these scripts are configured to run from - Eclipse or Maven as pulled directly from the ESAPI Subversion - repository or from pulled down from the zip file on Google Code at: - http://code.google.com/p/owasp-esapi-java/downloads/list + Eclipse or Maven as pulled directly from the ESAPI GitHub + repository or from pulled down from the zip file on GitHub at: + https://github.com/ESAPI/esapi-java-legacy/archive/refs/heads/develop.zip If you get it from any other place, then all bets are off. -- Kevin Wall diff --git a/src/examples/scripts/compile.sh b/src/examples/scripts/compile.sh index 95203d72b..eb5aa33e0 100755 --- a/src/examples/scripts/compile.sh +++ b/src/examples/scripts/compile.sh @@ -2,11 +2,11 @@ if [[ -z "$esapi_classpath" ]] then - echo 2>&1 "esapi_classpath not set. Did you dot the appropriate env file?" - echo 2>&1 "If you are using ESAPI from downloaded zip file, use:" - echo 2>&1 " . ./setenv-zip.sh" - echo 2>&1 "If you are using ESAPI pulled from SVN repository, use:" - echo 2>&1 " . ./setenv-svn.sh" + echo >&2 "esapi_classpath not set. Did you dot the appropriate env file?" + echo >&2 "If you are using ESAPI from downloaded zip file, use:" + echo >&2 " . ./setenv-zip.sh" + echo >&2 "If you are using ESAPI pulled from SVN repository, use:" + echo >&2 " . ./setenv-git.sh" exit 1 fi cd ../java diff --git a/src/examples/scripts/encryptProperties.sh b/src/examples/scripts/encryptProperties.sh index 4ee001d59..7b09e3985 100755 --- a/src/examples/scripts/encryptProperties.sh +++ b/src/examples/scripts/encryptProperties.sh @@ -9,11 +9,11 @@ USAGE="Usage: encryptedProperties.sh {-display|-create} [encrypted_properties_fi if [[ -z "$esapi_classpath" ]] then - echo 2>&1 "esapi_classpath not set. Did you dot the appropriate env file?" - echo 2>&1 "If you are using ESAPI from downloaded zip file, use:" - echo 2>&1 " . ./setenv-zip.sh" - echo 2>&1 "If you are using ESAPI pulled from SVN repository, use:" - echo 2>&1 " . ./setenv-svn.sh" + echo >&2 "esapi_classpath not set. Did you dot the appropriate env file?" + echo >&2 "If you are using ESAPI from downloaded zip file, use:" + echo >&2 " . ./setenv-zip.sh" + echo >&2 "If you are using ESAPI pulled from SVN repository, use:" + echo >&2 " . ./setenv-git.sh" exit 1 fi @@ -60,11 +60,14 @@ else echo "The property value will be encrypted and the value will be in plaintext" echo "and they will be placed in the specified output file." echo "End entering key/value pairs by entering an empty key & value." + echo + echo "Using your TEST version of ESAPI.properties file: $esapi_resources_test/ESAPI.properties" echo =========================================================== echo echo "Hit to continue..."; read GO set -x java -Dorg.owasp.esapi.resources="$esapi_resources_test" \ + -Djava.util.logging.config.file="$esapi_resources/esapi-java-logging.properties" \ -classpath "$esapi_classpath" \ org.owasp.esapi.reference.crypto.DefaultEncryptedProperties "$filename" && echo "Output of encrypted properties in file: $filename" diff --git a/src/examples/scripts/persistEncryptedData.sh b/src/examples/scripts/persistEncryptedData.sh index fc4a5ed46..a63fce2c9 100755 --- a/src/examples/scripts/persistEncryptedData.sh +++ b/src/examples/scripts/persistEncryptedData.sh @@ -10,19 +10,21 @@ if [[ -z "$esapi_classpath" ]] then - echo 2>&1 "esapi_classpath not set. Did you dot the appropriate env file?" - echo 2>&1 "If you are using ESAPI from downloaded zip file, use:" - echo 2>&1 " . ./setenv-zip.sh" - echo 2>&1 "If you are using ESAPI pulled from SVN repository, use:" - echo 2>&1 " . ./setenv-svn.sh" + echo >&2 "esapi_classpath not set. Did you dot the appropriate env file?" + echo >&2 "If you are using ESAPI from downloaded zip file, use:" + echo >&2 " . ./setenv-zip.sh" + echo >&2 "If you are using ESAPI pulled from SVN repository, use:" + echo >&2 " . ./setenv-git.sh" exit 1 fi cd ../java -set -x # Since this is just an illustration, we will use the test ESAPI.properties in # $esapi_resources_test. That way, it won't matter if the user has neglected # to run the 'setMasterKey.sh' example before running this one. +echo "Using your TEST version of ESAPI.properties file: $esapi_resources_test/ESAPI.properties" +set -x java -Dorg.owasp.esapi.resources="$esapi_resources_test" \ + -Djava.util.logging.config.file="$esapi_resources/esapi-java-logging.properties" \ -ea -classpath "$esapi_classpath" \ PersistedEncryptedData "$@" diff --git a/src/examples/scripts/runClass.sh b/src/examples/scripts/runClass.sh index 2b28a4a4c..c8fbf4b8e 100755 --- a/src/examples/scripts/runClass.sh +++ b/src/examples/scripts/runClass.sh @@ -3,11 +3,11 @@ if [[ -z "$esapi_classpath" ]] then - echo 2>&1 "esapi_classpath not set. Did you dot the appropriate env file?" - echo 2>&1 "If you are using ESAPI from downloaded zip file, use:" - echo 2>&1 " . ./setenv-zip.sh" - echo 2>&1 "If you are using ESAPI pulled from SVN repository, use:" - echo 2>&1 " . ./setenv-svn.sh" + echo >&2 "esapi_classpath not set. Did you dot the appropriate env file?" + echo >&2 "If you are using ESAPI from downloaded zip file, use:" + echo >&2 " . ./setenv-zip.sh" + echo >&2 "If you are using ESAPI pulled from SVN repository, use:" + echo >&2 " . ./setenv-git.sh" exit 1 fi @@ -18,9 +18,10 @@ if [[ ! -r ${className}.class ]] then echo >2&1 "Can't find class file: ${className}.class" exit 1 fi -echo "Your ESAPI.properties file: ${esapi_resources_test:?}/ESAPI.properties" +echo "Using your TEST version of ESAPI.properties file: ${esapi_resources_test:?}/ESAPI.properties" echo set -x java -Dorg.owasp.esapi.resources="$esapi_resources_test" \ + -Djava.util.logging.config.file="$esapi_resources/esapi-java-logging.properties" \ -classpath "$esapi_classpath" \ ${className} "$@" diff --git a/src/examples/scripts/setMasterKey.sh b/src/examples/scripts/setMasterKey.sh index ecd0937f7..74b1fa3ae 100755 --- a/src/examples/scripts/setMasterKey.sh +++ b/src/examples/scripts/setMasterKey.sh @@ -1,21 +1,29 @@ #!/bin/sh -if [[ -z "$esapi_classpath" ]] +if [ -z "$esapi_classpath" ] then - echo 2>&1 "esapi_classpath not set. Did you dot the appropriate env file?" - echo 2>&1 "If you are using ESAPI from downloaded zip file, use:" - echo 2>&1 " . ./setenv-zip.sh" - echo 2>&1 "If you are using ESAPI pulled from SVN repository, use:" - echo 2>&1 " . ./setenv-svn.sh" + echo >&2 "esapi_classpath not set. Did you dot the appropriate env file?" + echo >&2 "If you are using ESAPI from downloaded zip file, use:" + echo >&2 " . ./setenv-zip.sh" + echo >&2 "If you are using ESAPI pulled from Git or Svn repository, use:" + echo >&2 " . ./setenv-git.sh" exit 1 fi cd ../java -echo "Your ESAPI.properties file: $esapi_properties" +echo "This will generate the properties Encryptor.MasterKey and Encryptor.MasterSalt" +echo "which you will have to paste into your production ESAPI.properties file." echo -# set -x +echo "Do NOT copy those properties from your TEST ESAPI.properties as they are" +echo "the same for everyone and therefore are not secret." +echo +echo "Your PRODUCTION version of ESAPI.properties file: $esapi_resources/ESAPI.properties" +echo "Hit to continue..."; read GO +echo +set -x # This should use the real ESAPI.properties in $esapi_resources that does # not yet have Encryptor.MasterKey and Encryptor.MasterSalt yet set. java -Dorg.owasp.esapi.resources="$esapi_resources" \ + -Djava.util.logging.config.file="$esapi_resources/esapi-java-logging.properties" \ -classpath "$esapi_classpath" \ org.owasp.esapi.reference.crypto.JavaEncryptor "$@" diff --git a/src/examples/scripts/setenv-git.sh b/src/examples/scripts/setenv-git.sh new file mode 100755 index 000000000..99b9a86f8 --- /dev/null +++ b/src/examples/scripts/setenv-git.sh @@ -0,0 +1,40 @@ +#/bin/bash +# Purpose: Use to set up environment to compile and run examples if ESAPI +# downloaded from an Svn or Git repository. +# Usage: From csh, tcsh: +# $ source ./setenv-git.sh +# From most other *nix shells: +# $ . ./setenv-git.sh +# +# where '$' represents the shell command line prompt. +########################################################################### + +esapi_resources="$(\cd ../../../configuration/esapi >&- 2>&- && pwd)" +esapi_resources_test="$(\cd ../../../src/test/resources/esapi >&- 2>&- && pwd)" + + +# IMPORTANT NOTE: These dependency versions may need updated. Should match +# what is in ESAPI's pom.xml. +esapi_classpath=".:\ +../../../target/classes:\ +$(ls ../../../target/esapi-*.jar 2>&- || echo .):\ +$(./findjar.sh commons-fileupload-1.5.jar):\ +$(./findjar.sh servlet-api-2.4.jar)" + +if [[ ! -r "$esapi_resources"/ESAPI.properties ]] +then echo >&2 "setenv-git.sh: Can't read ESAPI.properties in $esapi_resources" + return 1 # Don't use 'exit' here or it will kill their current shell. +fi + +if [[ ! -r "$esapi_resources_test"/ESAPI.properties ]] +then echo >&2 "setenv-git.sh: Can't read ESAPI.properties in $esapi_resources_test" + return 1 # Don't use 'exit' here or it will kill their current shell. +fi + +echo ############################################################ +echo "esapi_resources=$esapi_resources" +echo "esapi_resources_test=$esapi_resources_test" +echo "esapi_classpath=$esapi_classpath" +echo ############################################################ + +export esapi_classpath esapi_resources esapi_resources_test diff --git a/src/examples/scripts/setenv-svn.sh b/src/examples/scripts/setenv-svn.sh index d8ab6d727..a60a498cf 100755 --- a/src/examples/scripts/setenv-svn.sh +++ b/src/examples/scripts/setenv-svn.sh @@ -22,12 +22,12 @@ esapi_resources_test="$(\cd ../../../src/test/resources/esapi >&- 2>&- && pwd)" if [[ ! -r "$esapi_resources"/ESAPI.properties ]] -then echo 2>&1 "setenv-svn.sh: Can't read ESAPI.properties in $esapi_resources" +then echo >&2 "setenv-svn.sh: Can't read ESAPI.properties in $esapi_resources" return 1 # Don't use 'exit' here or it will kill their current shell. fi if [[ ! -r "$esapi_resources_test"/ESAPI.properties ]] -then echo 2>&1 "setenv-svn.sh: Can't read ESAPI.properties in $esapi_resources_test" +then echo >&2 "setenv-svn.sh: Can't read ESAPI.properties in $esapi_resources_test" return 1 # Don't use 'exit' here or it will kill their current shell. fi diff --git a/src/examples/scripts/setenv-zip.sh b/src/examples/scripts/setenv-zip.sh index 199d9c055..8bf714c97 100755 --- a/src/examples/scripts/setenv-zip.sh +++ b/src/examples/scripts/setenv-zip.sh @@ -9,6 +9,9 @@ # where '$' represents the shell command line prompt. ########################################################################### +esapi_resources="$(\cd ../../../configuration/esapi >&- 2>&- && pwd)" +esapi_resources_test="$(\cd ../../../src/test/resources/esapi >&- 2>&- && pwd)" + # Here we don't look for the specific versions of the dependent libraries # since the specific version of the library is delivered as part of the # ESAPI zip file. In this manner, we do not have to update this if these @@ -18,17 +21,13 @@ $(ls ../../../esapi*.jar):\ $(./findjar.sh -start ../../../libs commons-fileupload-*.jar):\ $(./findjar.sh -start ../../../libs servlet-api-*.jar)" -esapi_resources="$(\cd ../../../configuration/esapi >&- 2>&- && pwd)" -esapi_resources_test="$(\cd ../../../src/test/resources/esapi >&- 2>&- && pwd)" - - if [[ ! -r "$esapi_resources"/ESAPI.properties ]] -then echo 2>&1 "setenv-svn.sh: Can't read ESAPI.properties in $esapi_resources" +then echo >&2 "setenv-zip.sh: Can't read ESAPI.properties in $esapi_resources" return 1 # Don't use 'exit' here or it will kill their current shell. fi if [[ ! -r "$esapi_resources_test"/ESAPI.properties ]] -then echo 2>&1 "setenv-svn.sh: Can't read ESAPI.properties in $esapi_resources_test" +then echo >&2 "setenv-zip.sh: Can't read ESAPI.properties in $esapi_resources_test" return 1 # Don't use 'exit' here or it will kill their current shell. fi diff --git a/src/main/java/org/owasp/esapi/HTTPUtilities.java b/src/main/java/org/owasp/esapi/HTTPUtilities.java index faa950dcc..9cdfc36b0 100644 --- a/src/main/java/org/owasp/esapi/HTTPUtilities.java +++ b/src/main/java/org/owasp/esapi/HTTPUtilities.java @@ -30,7 +30,12 @@ /** * The HTTPUtilities interface is a collection of methods that provide additional security related to HTTP requests, * responses, sessions, cookies, headers, and logging. - * + *

+ * Note: This most of the methods in this interface NOT compatible with the Jakarta Servlet API Spec 5.0 or later, which + * uses the jakarta.servlet package namespace rather than the javax.servlet package namespace. For further details, + * please see the GitHub Discussion issue + * Add support for Jakarta Servlet API Specification #768. + *

* @author Jeff Williams (jeff.williams .at. aspectsecurity.com) Aspect Security * @since June 1, 2007 */ @@ -274,7 +279,7 @@ public interface HTTPUtilities HttpServletResponse getCurrentResponse(); /** - * Calls getFileUploads with the *current* request, default upload directory, and default allowed file extensions + * Calls {@code getFileUploads} with the current request, default upload directory, and default allowed file extensions * * @return List of new File objects from upload * @throws ValidationException if the file fails validation @@ -284,7 +289,9 @@ public interface HTTPUtilities List getFileUploads() throws ValidationException; /** - * Call getFileUploads with the specified request, default upload directory, and default allowed file extensions + * Call {@code getFileUploads} with the specified request, default upload directory, and default allowed file extensions + * + * @param request The applicable HTTP request * * @return List of new File objects from upload * @throws ValidationException if the file fails validation @@ -292,7 +299,10 @@ public interface HTTPUtilities List getFileUploads(HttpServletRequest request) throws ValidationException; /** - * Call getFileUploads with the specified request, specified upload directory, and default allowed file extensions + * Call {@code getFileUploads} with the specified request, specified upload directory, and default allowed file extensions + * + * @param request The applicable HTTP request + * @param finalDir The destination directory to leave the uploaded file(s) in. * * @return List of new File objects from upload * @throws ValidationException if the file fails validation @@ -301,16 +311,87 @@ public interface HTTPUtilities /** - * Extract uploaded files from a multipart HTTP requests. Implementations must check the content to ensure that it + * Extract uploaded files from a multipart/form-data HTTP request. Implementations must check the content to ensure that it * is safe before making a permanent copy on the local filesystem. Checks should include length and content checks, * possibly virus checking, and path and name checks. Refer to the file checking methods in Validator for more - * information. - *

- * This method uses {@link HTTPUtilities#getCurrentRequest()} to obtain the {@link HttpServletRequest} object - * - * @param request - * @return List of new File objects from upload + * information. Important Note: The ESAPI reference implementation (i.e., + * {@code org.owasp.esapi.referenceDefaultHTTPUtilities} only does some of these things listed above and some of those + * are limited to which {@code getFileUploads} method is called and how you've set your relevant ESAPI properties + * in your ESAPI.properties file. + *

+ * This method uses {@link HTTPUtilities#getCurrentRequest()} to obtain the + * {@link javax.servlet.http.HttpServletRequest HttpServletRequest} + * object. If the ESAPI property HttpUtilities.FileUploadAllowAnonymousUser is set to {@code false} (the + * default is {@code true}), then {@code getFileUploads} will call {@code ESAPI.authenticator().getCurrentUser()} + * to check if the user is authenticated. If that property is set to {@code false} and a call to that function returns + * an anonymous (i.e., unauthenticated) user, then the file upload is blocked. + *

+ * ESAPI properties relevant to this and the other {@code getFileUploads} methods referenced in this table. The + * last 2 properties are new since release 2.5.2.0: + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + *
ESAPI Property NameESAPI.properties DefaultBuiltin DefaultMeaning
HttpUtilities.UploadDirC:\ESAPI\testUploadUploadDirFinal destination directory for uploaded files.
HttpUtilities.UploadTempDirC:\tempValue of system property java.io.tmpdirTemporary staging directory for uploaded files.
HttpUtilities.ApprovedUploadExtensions.pdf,.doc,.docx,.ppt,.pptx,.xls,.xlsx,.rtf,.txt,.jpg,.pn.pdf,.txt,.jpg,.pngComma separated allowed list of file suffixes that may be uploaded.
HttpUtilities.MaxUploadFileBytes50000005000000Total maximum upload file size for uploaded files per HTTP request.
HttpUtilities.MaxUploadFileCount2020Maximum total number of uploaded files per HTTP request.
HttpUtilities.FileUploadAllowAnonymousUsertruetrueControls whether anonymous (i.e., unauthenticated) users may upload files.
+ *

+ * As alluded to above, it is important to note that these {@code getFileUploads} methods do not do + * everything to keey your application and environment secure. Some of the more obvious omissions are the + * absence of examining the actual file content to determine the actual file type or running some AV scan + * on the uploaded files. You have to add that functionality to you if you want or need that. Some + * reasource that you may find usefule are: + *

+ * + * @param request The applicable HTTP request + * @param destinationDir The destination directory to leave the uploaded file in. + * @param allowedExtensions Permitted file suffixes. (Yes, this is a weak check. Use Apache Tika if you + * want something more.) + * @return List of new {@code File} objects from upload * @throws ValidationException if the file fails validation + * @throws java.security.AccessControlException If anonymous users are not allowed and the user is + * not authenticated as per the ESAPI {@code Authenticator}. */ List getFileUploads(HttpServletRequest request, File destinationDir, List allowedExtensions) throws ValidationException; diff --git a/src/main/java/org/owasp/esapi/PropNames.java b/src/main/java/org/owasp/esapi/PropNames.java index 59deb8991..2f3f8ee49 100644 --- a/src/main/java/org/owasp/esapi/PropNames.java +++ b/src/main/java/org/owasp/esapi/PropNames.java @@ -100,6 +100,8 @@ public final class PropNames { public static final String UPLOAD_TEMP_DIRECTORY = "HttpUtilities.UploadTempDir"; public static final String APPROVED_UPLOAD_EXTENSIONS = "HttpUtilities.ApprovedUploadExtensions"; public static final String MAX_UPLOAD_FILE_BYTES = "HttpUtilities.MaxUploadFileBytes"; + public static final String MAX_UPLOAD_FILE_COUNT = "HttpUtilities.MaxUploadFileCount"; + public static final String FILEUPLOAD_ALLOW_ANONYMOUS_USERS = "HttpUtilities.FileUploadAllowAnonymousUser"; public static final String RESPONSE_CONTENT_TYPE = "HttpUtilities.ResponseContentType"; public static final String HTTP_SESSION_ID_NAME = "HttpUtilities.HttpSessionIdName"; diff --git a/src/main/java/org/owasp/esapi/User.java b/src/main/java/org/owasp/esapi/User.java index fa30ca3ba..6c29b4f59 100644 --- a/src/main/java/org/owasp/esapi/User.java +++ b/src/main/java/org/owasp/esapi/User.java @@ -417,7 +417,7 @@ public interface User extends Principal, Serializable { * always a real user, the ANONYMOUS user is better than using null to represent * this. */ - User ANONYMOUS = new User() { + final User ANONYMOUS = new User() { private static final long serialVersionUID = -1850916950784965502L; diff --git a/src/main/java/org/owasp/esapi/ValidationRule.java b/src/main/java/org/owasp/esapi/ValidationRule.java index 3f9c3d42f..6c462216c 100644 --- a/src/main/java/org/owasp/esapi/ValidationRule.java +++ b/src/main/java/org/owasp/esapi/ValidationRule.java @@ -4,6 +4,11 @@ import org.owasp.esapi.errors.ValidationException; +/** + * A ValidationRule performs syntax and possibly semantic validation of a single piece of data from an untrusted source. + * + * @since ESAPI 2.0GA + */ public interface ValidationRule { /** diff --git a/src/main/java/org/owasp/esapi/package.html b/src/main/java/org/owasp/esapi/package.html index 02df5ede7..a5e2252c3 100644 --- a/src/main/java/org/owasp/esapi/package.html +++ b/src/main/java/org/owasp/esapi/package.html @@ -44,8 +44,8 @@

Sponsor

ESAPI Architecture

The ESAPI class library builds on the excellent security libraries available, -such as Java Logging, JCE, and Adobe Commons FileUpload. It uses the -concepts from many of the security packages out there, such as ACEGI, +such as Java Logging, JCE, and Apache Commons FileUpload. It uses the +concepts from many of the security packages out there, such as Spring Security, Apache Commons Validator, Microsoft's AntiXSS library, and many many more. This library provides a single consistent interface to security functions that is intuitive for enterprise developers.

diff --git a/src/main/java/org/owasp/esapi/reference/DefaultHTTPUtilities.java b/src/main/java/org/owasp/esapi/reference/DefaultHTTPUtilities.java index 53fe6135e..d652e6d56 100644 --- a/src/main/java/org/owasp/esapi/reference/DefaultHTTPUtilities.java +++ b/src/main/java/org/owasp/esapi/reference/DefaultHTTPUtilities.java @@ -41,6 +41,7 @@ import org.owasp.esapi.ESAPI; import org.owasp.esapi.HTTPUtilities; import org.owasp.esapi.Logger; +import org.owasp.esapi.PropNames; import org.owasp.esapi.SecurityConfiguration; import org.owasp.esapi.StringUtilities; import org.owasp.esapi.User; @@ -50,6 +51,7 @@ import org.owasp.esapi.crypto.PlainText; import org.owasp.esapi.errors.AccessControlException; import org.owasp.esapi.errors.AuthenticationException; +import org.owasp.esapi.errors.ConfigurationException; import org.owasp.esapi.errors.EncodingException; import org.owasp.esapi.errors.EncryptionException; import org.owasp.esapi.errors.IntegrityException; @@ -139,6 +141,36 @@ public void setResponse(HttpServletResponse newResponse) { /** The max bytes. */ static final int maxBytes = ESAPI.securityConfiguration().getAllowedFileUploadSize(); + /** The max # of files per request. */ + static int maxFiles = 20; // Same as default in configuration/esapi/ESAPI.properties + + static boolean fileUploadAllowAnonymousUsers = true; + + static { + // OPENISSUE - Not sure if we should log this. I can throw because the + // property is not set in ESAPI.properties, but it can also throw + // because ESAPI.properties can't be found. If the latter is the case, + // then trying to log it would cause another ConfigurationException to + // be thrown while trying do the logging making the exception stack + // traces even more obtuse. And I don't want to spend then next 5 years + // answering Stack Overflow questions about that. + try { + maxFiles = ESAPI.securityConfiguration().getIntProp( PropNames.MAX_UPLOAD_FILE_COUNT ); + } catch ( ConfigurationException ex ) { + // TODO: Figure out what we want to do log this. See OPENISSUE, above. + System.err.println("WARNING: Caught exception looking for property " + PropNames.MAX_UPLOAD_FILE_COUNT + + " in ESAPI.properties. Using hard-coded default of " + maxFiles + + "; exception was: " + ex); + } + try { + fileUploadAllowAnonymousUsers = ESAPI.securityConfiguration().getBooleanProp( PropNames.FILEUPLOAD_ALLOW_ANONYMOUS_USERS ); + } catch ( ConfigurationException ex ) { + // This likely will be the normal case (because ESAPI clients seldom update + // their ESAPI.properties file from release to release. Therefore, I am + // going to ignore it, and we silently go with the default of 'false'. + ; // Intentionally ignore! + } + } /* * The currentRequest ThreadLocal variable is used to make the currentRequest available to any call in any part of an @@ -159,7 +191,7 @@ public void setResponse(HttpServletResponse newResponse) { /** * No arg constructor. */ - public DefaultHTTPUtilities() { + public DefaultHTTPUtilities() { // Public CTOR for singletons. SMH. Sigh. } @@ -545,6 +577,16 @@ public List getFileUploads(HttpServletRequest request, File finalDir, List finalDir = ESAPI.securityConfiguration().getUploadDirectory(); } + // Check if this user should be blocked from file upload access. See allowUserFileUploadAccess() and comments + // around 'HttpUtilities.FileUploadAllowAnonymousUser' in ESAPI.properties file for details. + if ( ! allowUserFileUploadAccess() ) { + final String authZErrorMsg = "Upload failed. Anonymous user disallowed by ESAPI property " + + "'HttpUtilities.FileUploadAllowAnonymousUser'; attempted file upload blocked."; + logger.warning(Logger.SECURITY_FAILURE, authZErrorMsg); + throw new java.security.AccessControlException( authZErrorMsg ); + } + + List newFiles = new ArrayList(); String dfiPrevValue = "false"; // Fail safely in case of Java Security Manager & weird security policy try { @@ -555,11 +597,11 @@ public List getFileUploads(HttpServletRequest request, File finalDir, List throw new ValidationUploadException("Upload failed", "Not a multipart request"); } - // this factory will store ALL files in the temp directory, - // regardless of size + // this factory will store ALL files in the temp directory, regardless of size DiskFileItemFactory factory = new DiskFileItemFactory(0, tempDir); ServletFileUpload upload = new ServletFileUpload(factory); upload.setSizeMax(maxBytes); + upload.setFileCountMax(maxFiles); // Required to address CVE-2023-24998. // Create a progress listener ProgressListener progressListener = new ProgressListener() { @@ -1097,5 +1139,41 @@ private String decryptString(String ciphertext) throws EncryptionException { PlainText plaintext = ESAPI.encryptor().decrypt(restoredCipherText); return plaintext.toString(); } -} + /* Helper method, currently used only to check if we allow anonymous + * (i.e., unauthenticated) users to perform file uploads. + * The default is do allow anonymous users. We start to see if the property + * specified by "HttpUtilities.FileUploadAllowAnonymousUser" is set to + * {@code true} (which is the default). If that is true, we immediately + * return true as no further authentication check is required. However, if + * that property is false, then we check if the user is authenticated (i.e., + * the returned {@code User} object is not null). + */ + private boolean allowUserFileUploadAccess() { + if ( fileUploadAllowAnonymousUsers ) { + return true; // Okay to allow anonymous users + } + + // Check if the current user is authenticated as per ESAPI Authenticator. + User currentUser = ESAPI.authenticator().getCurrentUser(); + + if ( ! currentUser.isAnonymous() ) { + // + // The logging here is set to 'debug' rather than 'info' + // because I figured that the web / app server access logs + // should pick up the the authenticated user and log it + // along with whatever the HTTP request is, presumably including + // the authenticated user account name. + // + // If you are using JUL for logging and want to see this debug + // output, you need to change both instances of 'INFO' in your + // 'esapi-java-logging.properties' from 'INFO' to 'FINE' (not + // 'DEBUG', which isn't a thing in JUL). + // + logger.debug(Logger.SECURITY_SUCCESS, "Allowing authenticated user '" + currentUser.getAccountName() + + " to upload file(s) via getFileUploads method."); + return true; // User is authenticated, so allow access. + } + return false; // User not authenticated; deny access. + } +} diff --git a/src/main/java/org/owasp/esapi/waf/ESAPIWebApplicationFirewallFilter.java b/src/main/java/org/owasp/esapi/waf/ESAPIWebApplicationFirewallFilter.java index bd5e7f9a3..c1e7734f6 100644 --- a/src/main/java/org/owasp/esapi/waf/ESAPIWebApplicationFirewallFilter.java +++ b/src/main/java/org/owasp/esapi/waf/ESAPIWebApplicationFirewallFilter.java @@ -137,7 +137,7 @@ public void init(FilterConfig fc) throws ServletException { */ this.fc = fc; - logger.debug(Logger.EVENT_SUCCESS, ">> Initializing WAF"); + logger.debug(Logger.EVENT_SUCCESS, ">> Initializing ESAPI WAF"); /* * Pull logging file. -- We now ignore this arg, but will log something * letting users know we are ignoring it, because many of them never diff --git a/src/test/java/org/owasp/esapi/reference/HTTPUtilitiesTest.java b/src/test/java/org/owasp/esapi/reference/HTTPUtilitiesTest.java index 80a701021..2b164dbd9 100644 --- a/src/test/java/org/owasp/esapi/reference/HTTPUtilitiesTest.java +++ b/src/test/java/org/owasp/esapi/reference/HTTPUtilitiesTest.java @@ -41,6 +41,7 @@ import org.owasp.esapi.errors.EncryptionException; import org.owasp.esapi.errors.EnterpriseSecurityException; import org.owasp.esapi.errors.ValidationException; +import org.owasp.esapi.errors.ValidationUploadException; import org.owasp.esapi.http.MockHttpServletRequest; import org.owasp.esapi.http.MockHttpServletResponse; import org.owasp.esapi.http.MockHttpSession; @@ -52,6 +53,7 @@ import junit.framework.TestSuite; import static org.hamcrest.CoreMatchers.is; import static org.junit.Assert.assertThat; +// import org.junit.Ignore; // Doesn't seem to work with TestSuite. import org.junit.Rule; import org.junit.rules.ExpectedException; /** @@ -203,7 +205,7 @@ public void testChangeSessionIdentifier() throws EnterpriseSecurityException { } /** - * Test of formatHttpRequestForLog method, of class org.owasp.esapi.HTTPUtilities. + * Test of getFileUploads() method, of class org.owasp.esapi.HTTPUtilities. * @throws IOException */ public void testGetFileUploads() throws Exception { @@ -264,6 +266,115 @@ public void testGetFileUploads() throws Exception { } + /** + * Second test of getFileUpload() method, of class org.owasp.esapi.HTTPUtilities. + * This one is designed to fail by uploading 3 files. (The max is set to 2 files.) + * Based on experimentation with a dummy HTML form to send to localhost:8081 + * and the captured request caught by running 'nc -l 127.0.0.01 8081', this 'content' + * is what it looks like (changing the boundary back 'ridiculous') for the + * result of this HTML form: + * 
+     *    
+     *    
+     *    
+     *        Multifle-upload
+     *    
+     *    
+     *    Upload files...
+     *    

+     *       

+     *       What is your name?
+     *       

+     *       
What files are you sending?

+     *       
+     *       

+     *       
+     *       

+     *       
+     *       

+     *       

+     *       
+     *       
+     *       

+     *     

+     *    
+     *    
+     *
+ * with the 'full-name' field filled in with 'kevin w. wall' and the 3 + * uploaded files filled in with files named 'aaa.txt', 'bbb.txt', and 'ccc.txt', + * respectively and each those file containing created thusly from bash: + * 
+     *     $ echo AAA >aaa.txt
+     *     $ echo BBB >bbb.txt
+     *     $ echo CCC >ccc.txt
+     *
+ * Because we are uploading 3 files, but have the property HttpUtilities.MaxUploadFileCount + * set to 2 in 'src/test/resources/esapi/ESAPI.properties', the file upload + * attempt via HTTPUtilities.getFileUploads() will result in throwing a ValidationUploadException, + * and if you look through the exception stack trace, you can see the + * 'Caused by' reason is: + * Caused by: org.apache.commons.fileupload.FileCountLimitExceededException: attachment + * at org.apache.commons.fileupload.FileUploadBase.parseRequest(FileUploadBase.java:367) + * at org.apache.commons.fileupload.servlet.ServletFileUpload.parseRequest(ServletFileUpload.java:113) + * at org.owasp.esapi.reference.DefaultHTTPUtilities.getFileUploads(DefaultHTTPUtilities.java:628) + * ... 23 more + * which is as it should be. + * + */ + public void testGetFileUploadsTooManyFiles() throws Exception { + File home = null; + + System.out.println("testGetFileUploadsTooManyFiles"); + + try + { + home = FileTestUtils.createTmpDirectory(CLASS_NAME); + String content = "Content-Type: multipart/form-data; boundary=ridiculous\r\n\r\n\r\n" + + "--ridiculous\r\n" + + "Content-Disposition: form-data; name=\"full-name\"\r\n\r\n" + + "kevin w wall\r\n" + + "--ridiculous\r\n" + + "Content-Disposition: form-data; name=\"file1\"; filename=\"aaa.txt\"\r\n" + + "Content-Type: text/plain\r\n\r\n" + + "AAA\r\n\r\n" + + "--ridiculous\r\n" + + "Content-Disposition: form-data; name=\"file2\"; filename=\"bbb.txt\"\r\n" + + "Content-Type: text/plain\r\n\r\n" + + "BBB\r\n\r\n" + + "--ridiculous\r\n" + + "Content-Disposition: form-data; name=\"file3\"; filename=\"ccc.txt\"\r\n" + + "Content-Type: text/plain\r\n\r\n" + + "CCC\r\n\r\n" + + "--ridiculous--\r\n\r\n"; + + MockHttpServletResponse response = new MockHttpServletResponse(); + MockHttpServletRequest request1 = new MockHttpServletRequest("/test", content.getBytes(response.getCharacterEncoding())); + ESAPI.httpUtilities().setCurrentHTTP(request1, response); + + MockHttpServletRequest request2 = new MockHttpServletRequest("/test", content.getBytes(response.getCharacterEncoding())); + request2.setContentType( "multipart/form-data; boundary=ridiculous"); + ESAPI.httpUtilities().setCurrentHTTP(request2, response); + List response2 = new ArrayList<>(); + boolean caughtExpectedException = false; + try { + response2 = ESAPI.httpUtilities().getFileUploads(request2, home); + } catch( ValidationUploadException vuex ) { + caughtExpectedException = true; + } finally { + response2.forEach(file -> file.delete()); + } + // If this assertion fails, check the property HttpUtilities.MaxUploadFileCount in + // 'src/test/resources/esapi/ESAPI.properties' to make sure it is still to 2. + assertTrue("Did not catch expected ValidationUploadException because too many files uploaded.", caughtExpectedException ); + } + finally + { + FileTestUtils.deleteRecursively(home); + } + + } /** @@ -577,5 +688,75 @@ public void testGetRequestAttribute() throws Exception { Float test2 = ESAPI.httpUtilities().getRequestAttribute( request, "testAttribute" ); assertEquals( test2, 43f ); } -} + /** Test HTTPUtilities.getFileUploads with an unauthenticated (i.e., + * anonymous) user. In 'src/test/resources/esapi/ESAPI.properties', the + * property 'HttpUtilities.FileUploadAllowAnonymousUser' is set to 'false'. + * This is okay, because as it turns out most (all?) of these tests are + * executed after testCSRFToken(), which creates a users.txt file with + * a random user account name that gets used when + * 
+     *      ESAPI.authenticator().setCurrentUser( user );
+     *
+ * gets called a few lines later. That seems to persist throughout the + * remainder of this test suite. However, this test needs to clear that + * information so that any further HTTP requests are made as an anonymous + * user. + * + * However, there is a concern here. I is not clear whether or not this + * would have unintended consequences because I don't this assumptions can + * be made about the specific order these test cases within a test suite + * are executed in. + * + * Consequently, I ignoring this specific test by commenting it out for the + * concerns mentioned above. Unfortunately, the @Ignore annotation from + * JUnit 4 doesn't work here; apparently, it doesn't play nicely with the JUnit 3 + * construct of + * public static Test suite() { + * return new TestSuite(HTTPUtilitiesTest.class); + * } + * + * Note, however, the test does give the expected results and fails the + * upload as intended. + */ +/********************* KWWALL Commented Out - Do not delete this comment or test! ************* + public void testGetFileUploadsUnauthenticatedUser() throws Exception { + System.out.print( "testGetFileUploadsUnauthenticatedUser" ); + + File home = null; + + // Clear the current user info making it effective an anonymous user again. + ESAPI.authenticator().clearCurrent(); // Either this or logout(), but logout may kill the session too. + + try + { + home = FileTestUtils.createTmpDirectory(CLASS_NAME); + String content = "--ridiculous\r\nContent-Disposition: form-data; name=\"upload\"; filename=\"testupload.txt\"\r\nContent-Type: application/octet-stream\r\n\r\nThis is a test of the multipart broadcast system.\r\nThis is only a test.\r\nStop.\r\n\r\n--ridiculous\r\nContent-Disposition: form-data; name=\"submit\"\r\n\r\nSubmit Query\r\n--ridiculous--\r\nEpilogue"; + + MockHttpServletResponse response = new MockHttpServletResponse(); + MockHttpServletRequest request1 = new MockHttpServletRequest("/test", content.getBytes(response.getCharacterEncoding())); + ESAPI.httpUtilities().setCurrentHTTP(request1, response); + + MockHttpServletRequest request2 = new MockHttpServletRequest("/test", content.getBytes(response.getCharacterEncoding())); + request2.setContentType( "multipart/form-data; boundary=ridiculous"); + ESAPI.httpUtilities().setCurrentHTTP(request2, response); + List response2 = new ArrayList<>(); + try { + response2 = ESAPI.httpUtilities().getFileUploads(request2, home); + fail("Expecting an exception here"); + } catch ( java.security.AccessControlException acex ) { + ; // Expected + } catch ( Exception ex ) { + fail("Wrong exception type caught: " + ex.getClass().getName() + + ", received, expected java.security.AccessControlException"); + } finally { + response2.forEach(file -> file.delete()); + } + } + finally + { + FileTestUtils.deleteRecursively(home); + } + } +********************* KWWALL End Commented Out Code ********************/ +} diff --git a/src/test/resources/esapi/ESAPI.properties b/src/test/resources/esapi/ESAPI.properties index d9e12ce9c..9ab844622 100644 --- a/src/test/resources/esapi/ESAPI.properties +++ b/src/test/resources/esapi/ESAPI.properties @@ -373,6 +373,31 @@ HttpUtilities.httpQueryParamValueLength=500 # File upload configuration HttpUtilities.ApprovedUploadExtensions=.pdf,.doc,.docx,.ppt,.pptx,.xls,.xlsx,.rtf,.txt,.jpg,.png HttpUtilities.MaxUploadFileBytes=500000000 +# Maximum # of files that can be uploaded per HTTP request. +# Set to -1 for no maximum. Related to CVE-2023-24998. +# This is intentionally small for testing purposes. +HttpUtilities.MaxUploadFileCount=2 + +# Allowing anonymous users to do file uploads via HTTPUtilities.getFileUploads +# can make it easier for DoS attacks via uploading files easier. (See Security Bulletin #11, +# https://github.com/ESAPI/esapi-java-legacy/blob/develop/documentation/ESAPI-security-bulletin11.pdf +# for details). +# +# By default, we allow anonymous users to upload files because we can only rely on +# ESAPI.authenticator().getCurrentUser() to determine if a user associated +# with the current HTTP session is authenticated and almost no one uses the +# ESAPI Authenticator because the reference implementation is just a toy +# implementation and is not enterprise scalable. +# +# If you are using the ESAPI Authenticator (the ESAPI reference implementation +# or you've implemented your own custom one), then you can set this property value +# to 'false' to disallow anonymous (i.e., unauthenticated) users to upload +# files. However, if you are not using the ESAPI Authenticator, then you should +# probably leave this set to 'false', otherwise you will completely prevent the +# use of HTTPUtilities.getFileUploads methods. +# +HttpUtilities.FileUploadAllowAnonymousUser=false + # Using UTF-8 throughout your stack is highly recommended. That includes your database driver, # container, and any other technologies you may be using. Failure to do this may expose you # to Unicode transcoding injection attacks. Use of UTF-8 does not hinder internationalization.