Fix bug where internal unquoted whitespace truncates values #205
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
joho
force-pushed
the
fix-1.5-unquoted-internal-whitespace-regression
branch
from
98e2598 to
27959f3
Compare
silverwind
added a commit
to silverwind/godotenv
that referenced
this pull request
Oct 28, 2024
* rename module path (silverwind) * REVERTME: rename forked module to avoid using replace directive in go.mod (Andrey Novikov) * Merge branch 'fix/expand-existing-var' (Andrey Novikov) * Re-add global env variable substitution (Andrey Novikov) * Fix panic because of wrong function (joho#223) (Kha Nguyen) * Update cmd.go (joho#221) (Vintage) * Bump actions/setup-go from 3 to 4 (joho#207) (dependabot[bot]) * Add whitespace tests. (joho#210) (Chris Jones) * Fix bug where internal unquoted whitespace truncates values (joho#205) (John Barton) * Fix: ioutil.ReadAll() is deprecated, so removed it's dependency (joho#202) (Rakibul Yeasin) * add overload flag (joho#200) (2tef) * fix whitespace with gofmt (joho#203) (2tef) * Update CI to test go 1.20 (joho#201) (John Barton) * Multiline string support (joho#156) (John Barton) * fix tiny details (joho#199) (2tef) * Add punctuation to please godoc (joho#197) (John Barton) * sort go import in readme (joho#193) (Doarakko) * Remove power8 again as it wasn't fixed (John Barton) * README updates around contributing (John Barton) * Add darwin arm64 build (joho#174) (Elliot Murphy) * Bump actions/setup-go from 2 to 3 (joho#185) (dependabot[bot]) * Bump actions/checkout from 2 to 3 (joho#184) (dependabot[bot]) * Create codeql-analysis.yml (joho#186) (John Barton) * Remove renovate, add dependabot (joho#183) (John Barton) * Update README for go install post go 1.17 (joho#170) (Bahtiyar Biksayev) * Add a few new go versions (John Barton) * remove power8 tests as they stopped working (John Barton) * Fix typos in comments and extend README (joho#177) (Oleksandr Redko) * Fix CI for power8 arch (joho#157) (John Barton) * Add missing newline to created env files (joho#133) (Alexander Klein) * Add renovate.json (joho#124) (renovate[bot]) * Fix release name (John Barton) * Try a different github action (John Barton) * Add action to publish binaries on each release (joho#125) (John Barton) * Add a go.mod file (joho#123) (John Barton) * Non amd tests (joho#122) (John Barton) * Merge pull request joho#121 from joho/github-actions (John Barton) * Add CI badge to readme (John Barton) * Matrix test across mac/win/linux (John Barton) * Add Barebones github actions test (John Barton) * Merge pull request joho#115 from adombeck/master (John Barton) * Merge pull request joho#109 from mniak/pr (John Barton) * Merge pull request joho#120 from santosh653/master (John Barton) * Update .travis.yml (santosh653) * Update .travis.yml (santosh653) * Fix typo (adombeck) * add newline back (Andre Soares) * Fix package name (Andre Soares) * Remove go.mod (Andre Soares) * Change package name back to joho/godotenv (Andre Soares) * Rename and implement int-able without quotes (Andre Soares) * Merge pull request joho#95 from orxobo/Write-adjustment (John Barton) * Fixed Write bugs (orxobo) * Merge pull request joho#90 from djherbis/master (John Barton) * joho#89 move regexp.MustCompile to globals (Dustin H) * Merge pull request joho#69 from hairyhenderson/ignore-leading-whitespace (John Barton) * Merge branch 'master' into ignore-leading-whitespace (John Barton) * Merge pull request joho#70 from hairyhenderson/support-keys-beginning-with-export-66 (John Barton) * Support key names beginning with 'export' (Dave Henderson) * Ignoring leading whitespace (Dave Henderson) * Merge pull request joho#63 from mdanzinger/patch-1 (John Barton) * An even tinier grammar fix (John Barton) * Update cmd.go (Mendy Danzinger) * Merge pull request joho#60 from coolaj86/precedence (John Barton) * explicitly state env precedence and convention (AJ ONeal) * Merge pull request joho#58 from lucastetreault/master (John Barton) * feat(Expand Variables): Custom variable expansion instead of Go's os.Expand (Lucas Tétreault) * Merge pull request joho#54 from egorse/master (John Barton) * The value expand fallback to actual ENV values (egorse) * Merge pull request joho#47 from sachaos/feature-expand-variables-on-value (John Barton) * Update my homepage link (John Barton) * Fix test, `$` should not be escaped (Takumasa Sakao) * Expand variables on parseValue (Takumasa Sakao) * Add test for substitutions (Takumasa Sakao) * Pass envMap to parseLine & parseValue (Takumasa Sakao) * Merge pull request joho#46 from joho/fix_quoting_parser_bug (John Barton) * Try and replicate reported bug joho#45 (John Barton (joho)) * Merge pull request joho#44 from dvrkps/patch-1 (John Barton) * travis: update go version (Davor Kapsa) * Merge pull request joho#41 from alexquick/document-write-dotenv (John Barton) * Merge pull request joho#42 from alexquick/feature-sorted-output (John Barton) * sort output of Write/Marshal (Alex Quick) * Be more careful with TestRoundtrip (Alex Quick) * document Marshal, Unmarshal, and Write (Alex Quick) * Merge pull request joho#35 from alexquick/feature-write-dotenv (John Barton) * rename WriteString/ReadString to Marshal/Unmarshal (Alex Quick) * escape some other bash-y special chars ($!) (Alex Quick) * support for writing envs out in dotenv format (Alex Quick) * Merge pull request joho#34 from alexquick/fix-parsing-issues (John Barton) * Merge pull request joho#33 from crash7/go-report-card (John Barton) * Merge pull request joho#36 from pda/parse-from-reader (John Barton) * README.md mentions Parse(io.Reader) (Paul Annesley) * Parse(io.Reader) => map[string]string (Paul Annesley) * fix panic with `"` as the value (Alex Quick) * support for equals in yaml-style lines (Alex Quick) * rudimentry support for nested quotes (Alex Quick) * handle escaping more comprehensively (Alex Quick) * Add Go Report Card badge and fix spelling error (Christian Musa) * README housekeeping (John Barton) * Merge pull request joho#29 from joho/respect_empty_external_env_vars (John Barton) * Change check of existing env to respect empty (but set) vars. (John Barton (joho)) * Add failing test for override of empty var (John Barton (joho)) * Merge pull request joho#27 from goenning/empty_var (John Barton) * allow empty_var (Oenning) * Merge pull request joho#25 from matiasanaya/master (John Barton) * Fix quoted values check (Matias Anaya) * Merge pull request joho#24 from joho/setup_travis (John Barton) * Add a badge for the windows build too. (John Barton (joho)) * Apparently this file is meant to be hidden? (John Barton (joho)) * Switch build badge over to travis (John Barton (joho)) * Replace wercker.yml with travis.yml (John Barton (joho)) * Merge pull request joho#22 from mmilata/dont-swallow-errors (John Barton) * Don't hide line parsing errors (Martin Milata) * Propagate errors encountered when reading file (Martin Milata) * Merge pull request joho#13 from jmervine/master (John Barton) * adding Overload method (Josh Mervine) * Merge pull request joho#11 from buddhamagnet/master (John Barton) * remove unecessary assignment in autoloader (buddhamagnet) * outdent else because golint said so. (John Barton (joho)) * Still trying to please golint with package comments. (John Barton (joho)) * Some golint feedback from http://goreportcard.com/report/joho/godotenv (John Barton (joho)) * run `gofmt -w -s ./..` (John Barton (joho)) * Merge pull request joho#8 from calavera/add_values_to_envmap (John Barton) * Add values to the envMap when reading the file. (David Calavera) * Update docs for the bin command. (John Barton (joho)) * Move cmd.go so it installs as "godotenv" (John Barton (joho)) * Merge pull request joho#6 from joho/add_bin_command (John Barton) * Improve godoc on Read and Exec (John Barton (joho)) * Move the env loadings & command running into main package. (John Barton (joho)) * Get the minimum viable of the command working. (John Barton (joho)) * WIP on adding a bin command (John Barton (joho)) * Add CI badge for windows from Appveyor. (John Barton (joho)) * Remove commented out code and go fmt tests. (John Barton (joho)) * Merge pull request joho#4 from GoIncremental/4-windows-line-returns (John Barton) * fixup (David Bochenski) * Fix remaining failing test on windows (David Bochenski) * Closes joho#4 Use bufio.Scanner to read newLines (David Bochenski) * Merge pull request joho#3 from ys/feature/accept-equals-in-values (John Barton) * Use SplitN instead of split (Yannick Schutz) * Merge pull request joho#2 from mattn/seveal_equals (John Barton) * FOO=BAR= should be "FOO" = "BAR=" (mattn) * add godotenv.Read() which returns a map rather than calling os.Setenv (John Barton (joho)) * Merge pull request joho#1 from wolfeidau/readme_corrections (John Barton) * Small correction to code in the README. (Mark Wolfe) * Fix the autoload package. (John Barton (joho)) * Do more of the documenting. (John Barton (joho)) * Turns out there was an easier way to read the env file. (John Barton (joho)) * Add the ability to autoload the env file when you import. (John Barton (joho)) * Write up something for "go doc" (John Barton (joho)) * Update readme (John Barton (joho)) * Make sure we don't overwrite existing ENV vars. (John Barton (joho)) * That is a syntax lol. (John Barton (joho)) * Love me some build badges. (John Barton (joho)) * Borrow some of dotenv's readme (John Barton (joho)) * Licence and more readme (John Barton (joho)) * Readme with build status. (John Barton (joho)) * Add wercker.yml for CI (John Barton (joho)) * Make Load() without args open .env by default (John Barton (joho)) * Get the code running against all fixtures. (John Barton (joho)) * Clear the env at the start of each integration test. (John Barton (joho)) * Ignore comment lines and lines full o' whitespace. (John Barton (joho)) * Add some more tests. (John Barton (joho)) * Handle quoted hashes "properly" (John Barton (joho)) * Red, green, rinse repeat. (John Barton (joho)) * World's most naive comment parser... (John Barton (joho)) * Start trying to do comments/quoting properly. (John Barton (joho)) * Mark out the rest of my todo list in the test file. (John Barton (joho)) * export keyword parsing (John Barton (joho)) * Yaml style bro. (John Barton (joho)) * Move the red-green a little way down the existing parser specs. (John Barton (joho)) * Start porting over parser tests. (John Barton (joho)) * Extract a test helper (John Barton (joho)) * Get the basic tests green. (John Barton (joho)) * First failing tests extracted from dotenv (John Barton (joho)) * Macs are dumb. (John Barton (joho))
silverwind
added a commit
to silverwind/godotenv
that referenced
this pull request
Oct 28, 2024
* also replace in readme (silverwind) * 0.1.0 (silverwind) * rename module path (silverwind) * REVERTME: rename forked module to avoid using replace directive in go.mod (Andrey Novikov) * Merge branch 'fix/expand-existing-var' (Andrey Novikov) * Re-add global env variable substitution (Andrey Novikov) * Fix panic because of wrong function (joho#223) (Kha Nguyen) * Update cmd.go (joho#221) (Vintage) * Bump actions/setup-go from 3 to 4 (joho#207) (dependabot[bot]) * Add whitespace tests. (joho#210) (Chris Jones) * Fix bug where internal unquoted whitespace truncates values (joho#205) (John Barton) * Fix: ioutil.ReadAll() is deprecated, so removed it's dependency (joho#202) (Rakibul Yeasin) * add overload flag (joho#200) (2tef) * fix whitespace with gofmt (joho#203) (2tef) * Update CI to test go 1.20 (joho#201) (John Barton) * Multiline string support (joho#156) (John Barton) * fix tiny details (joho#199) (2tef) * Add punctuation to please godoc (joho#197) (John Barton) * sort go import in readme (joho#193) (Doarakko) * Remove power8 again as it wasn't fixed (John Barton) * README updates around contributing (John Barton) * Add darwin arm64 build (joho#174) (Elliot Murphy) * Bump actions/setup-go from 2 to 3 (joho#185) (dependabot[bot]) * Bump actions/checkout from 2 to 3 (joho#184) (dependabot[bot]) * Create codeql-analysis.yml (joho#186) (John Barton) * Remove renovate, add dependabot (joho#183) (John Barton) * Update README for go install post go 1.17 (joho#170) (Bahtiyar Biksayev) * Add a few new go versions (John Barton) * remove power8 tests as they stopped working (John Barton) * Fix typos in comments and extend README (joho#177) (Oleksandr Redko) * Fix CI for power8 arch (joho#157) (John Barton) * Add missing newline to created env files (joho#133) (Alexander Klein) * Add renovate.json (joho#124) (renovate[bot]) * Fix release name (John Barton) * Try a different github action (John Barton) * Add action to publish binaries on each release (joho#125) (John Barton) * Add a go.mod file (joho#123) (John Barton) * Non amd tests (joho#122) (John Barton) * Merge pull request joho#121 from joho/github-actions (John Barton) * Add CI badge to readme (John Barton) * Matrix test across mac/win/linux (John Barton) * Add Barebones github actions test (John Barton) * Merge pull request joho#115 from adombeck/master (John Barton) * Merge pull request joho#109 from mniak/pr (John Barton) * Merge pull request joho#120 from santosh653/master (John Barton) * Update .travis.yml (santosh653) * Update .travis.yml (santosh653) * Fix typo (adombeck) * add newline back (Andre Soares) * Fix package name (Andre Soares) * Remove go.mod (Andre Soares) * Change package name back to joho/godotenv (Andre Soares) * Rename and implement int-able without quotes (Andre Soares) * Merge pull request joho#95 from orxobo/Write-adjustment (John Barton) * Fixed Write bugs (orxobo) * Merge pull request joho#90 from djherbis/master (John Barton) * joho#89 move regexp.MustCompile to globals (Dustin H) * Merge pull request joho#69 from hairyhenderson/ignore-leading-whitespace (John Barton) * Merge branch 'master' into ignore-leading-whitespace (John Barton) * Merge pull request joho#70 from hairyhenderson/support-keys-beginning-with-export-66 (John Barton) * Support key names beginning with 'export' (Dave Henderson) * Ignoring leading whitespace (Dave Henderson) * Merge pull request joho#63 from mdanzinger/patch-1 (John Barton) * An even tinier grammar fix (John Barton) * Update cmd.go (Mendy Danzinger) * Merge pull request joho#60 from coolaj86/precedence (John Barton) * explicitly state env precedence and convention (AJ ONeal) * Merge pull request joho#58 from lucastetreault/master (John Barton) * feat(Expand Variables): Custom variable expansion instead of Go's os.Expand (Lucas Tétreault) * Merge pull request joho#54 from egorse/master (John Barton) * The value expand fallback to actual ENV values (egorse) * Merge pull request joho#47 from sachaos/feature-expand-variables-on-value (John Barton) * Update my homepage link (John Barton) * Fix test, `$` should not be escaped (Takumasa Sakao) * Expand variables on parseValue (Takumasa Sakao) * Add test for substitutions (Takumasa Sakao) * Pass envMap to parseLine & parseValue (Takumasa Sakao) * Merge pull request joho#46 from joho/fix_quoting_parser_bug (John Barton) * Try and replicate reported bug joho#45 (John Barton (joho)) * Merge pull request joho#44 from dvrkps/patch-1 (John Barton) * travis: update go version (Davor Kapsa) * Merge pull request joho#41 from alexquick/document-write-dotenv (John Barton) * Merge pull request joho#42 from alexquick/feature-sorted-output (John Barton) * sort output of Write/Marshal (Alex Quick) * Be more careful with TestRoundtrip (Alex Quick) * document Marshal, Unmarshal, and Write (Alex Quick) * Merge pull request joho#35 from alexquick/feature-write-dotenv (John Barton) * rename WriteString/ReadString to Marshal/Unmarshal (Alex Quick) * escape some other bash-y special chars ($!) (Alex Quick) * support for writing envs out in dotenv format (Alex Quick) * Merge pull request joho#34 from alexquick/fix-parsing-issues (John Barton) * Merge pull request joho#33 from crash7/go-report-card (John Barton) * Merge pull request joho#36 from pda/parse-from-reader (John Barton) * README.md mentions Parse(io.Reader) (Paul Annesley) * Parse(io.Reader) => map[string]string (Paul Annesley) * fix panic with `"` as the value (Alex Quick) * support for equals in yaml-style lines (Alex Quick) * rudimentry support for nested quotes (Alex Quick) * handle escaping more comprehensively (Alex Quick) * Add Go Report Card badge and fix spelling error (Christian Musa) * README housekeeping (John Barton) * Merge pull request joho#29 from joho/respect_empty_external_env_vars (John Barton) * Change check of existing env to respect empty (but set) vars. (John Barton (joho)) * Add failing test for override of empty var (John Barton (joho)) * Merge pull request joho#27 from goenning/empty_var (John Barton) * allow empty_var (Oenning) * Merge pull request joho#25 from matiasanaya/master (John Barton) * Fix quoted values check (Matias Anaya) * Merge pull request joho#24 from joho/setup_travis (John Barton) * Add a badge for the windows build too. (John Barton (joho)) * Apparently this file is meant to be hidden? (John Barton (joho)) * Switch build badge over to travis (John Barton (joho)) * Replace wercker.yml with travis.yml (John Barton (joho)) * Merge pull request joho#22 from mmilata/dont-swallow-errors (John Barton) * Don't hide line parsing errors (Martin Milata) * Propagate errors encountered when reading file (Martin Milata) * Merge pull request joho#13 from jmervine/master (John Barton) * adding Overload method (Josh Mervine) * Merge pull request joho#11 from buddhamagnet/master (John Barton) * remove unecessary assignment in autoloader (buddhamagnet) * outdent else because golint said so. (John Barton (joho)) * Still trying to please golint with package comments. (John Barton (joho)) * Some golint feedback from http://goreportcard.com/report/joho/godotenv (John Barton (joho)) * run `gofmt -w -s ./..` (John Barton (joho)) * Merge pull request joho#8 from calavera/add_values_to_envmap (John Barton) * Add values to the envMap when reading the file. (David Calavera) * Update docs for the bin command. (John Barton (joho)) * Move cmd.go so it installs as "godotenv" (John Barton (joho)) * Merge pull request joho#6 from joho/add_bin_command (John Barton) * Improve godoc on Read and Exec (John Barton (joho)) * Move the env loadings & command running into main package. (John Barton (joho)) * Get the minimum viable of the command working. (John Barton (joho)) * WIP on adding a bin command (John Barton (joho)) * Add CI badge for windows from Appveyor. (John Barton (joho)) * Remove commented out code and go fmt tests. (John Barton (joho)) * Merge pull request joho#4 from GoIncremental/4-windows-line-returns (John Barton) * fixup (David Bochenski) * Fix remaining failing test on windows (David Bochenski) * Closes joho#4 Use bufio.Scanner to read newLines (David Bochenski) * Merge pull request joho#3 from ys/feature/accept-equals-in-values (John Barton) * Use SplitN instead of split (Yannick Schutz) * Merge pull request joho#2 from mattn/seveal_equals (John Barton) * FOO=BAR= should be "FOO" = "BAR=" (mattn) * add godotenv.Read() which returns a map rather than calling os.Setenv (John Barton (joho)) * Merge pull request joho#1 from wolfeidau/readme_corrections (John Barton) * Small correction to code in the README. (Mark Wolfe) * Fix the autoload package. (John Barton (joho)) * Do more of the documenting. (John Barton (joho)) * Turns out there was an easier way to read the env file. (John Barton (joho)) * Add the ability to autoload the env file when you import. (John Barton (joho)) * Write up something for "go doc" (John Barton (joho)) * Update readme (John Barton (joho)) * Make sure we don't overwrite existing ENV vars. (John Barton (joho)) * That is a syntax lol. (John Barton (joho)) * Love me some build badges. (John Barton (joho)) * Borrow some of dotenv's readme (John Barton (joho)) * Licence and more readme (John Barton (joho)) * Readme with build status. (John Barton (joho)) * Add wercker.yml for CI (John Barton (joho)) * Make Load() without args open .env by default (John Barton (joho)) * Get the code running against all fixtures. (John Barton (joho)) * Clear the env at the start of each integration test. (John Barton (joho)) * Ignore comment lines and lines full o' whitespace. (John Barton (joho)) * Add some more tests. (John Barton (joho)) * Handle quoted hashes "properly" (John Barton (joho)) * Red, green, rinse repeat. (John Barton (joho)) * World's most naive comment parser... (John Barton (joho)) * Start trying to do comments/quoting properly. (John Barton (joho)) * Mark out the rest of my todo list in the test file. (John Barton (joho)) * export keyword parsing (John Barton (joho)) * Yaml style bro. (John Barton (joho)) * Move the red-green a little way down the existing parser specs. (John Barton (joho)) * Start porting over parser tests. (John Barton (joho)) * Extract a test helper (John Barton (joho)) * Get the basic tests green. (John Barton (joho)) * First failing tests extracted from dotenv (John Barton (joho)) * Macs are dumb. (John Barton (joho))
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reported in #204
The new parser for multiline values incorrectly truncates unquoted lines with internal whitespace.
There was no test for this previously so was "undefined" behaviour, but given we use https://github.com/bkeepers/dotenv as our "standard", it works there so it should work here.
Once getting deeper into the fix, I discovered not all tests had been moved over to the new parser and some dead code had been left in the package. I've now fixed all that up and got (almost all) original tests passing.
I cut two tests (allowing unterminated quoted variables) as they didn't match any rspec in ruby's dotenv.