Make sure pgtap errors get caught #46

vincentsarago · 2021-08-19T06:51:23Z

ref: #45

vincentsarago · 2021-08-19T06:59:38Z

I still don't get how the tests pass on GitHub while on my local machine the added tests returns

{"next": null, "prev": null, "type": "FeatureCollection", "context": {"limit": 10, "matched": 0, "returned": 0}, "features": null}

I'm not quite sure the pgtap tests are being run in CI at all 🤷‍♂️

vincentsarago · 2021-08-19T08:47:57Z

confirmed

the latest commit shows that the result of the pgtap command returns

Checking if any tests are not ok on db ***localhost:5432/migra_to
...
not ok 66 - Test basic search with fields and sort extension and next token
# Failed test 66: "Test basic search with fields and sort extension and next token"
#     Results differ beginning at row 1:
#         have: ("{""next"": ""pgstac-test-item-0020"", ""prev"": ""pgstac-test-item-0007"", ""type"": ""FeatureCollection"", ""context"": {""limit"": 10, ""matched"": 100, ""returned"": 10}, ""features"": [{""id"": ""pgstac-test-item-0007"", ""properties"": {""datetime"": ""2011-08-17T00:00:00Z"", ""eo:cloud_cover"": 59}}, {""id"": ""pgstac-test-item-0008"", ""properties"": {""datetime"": ""2011-08-17T00:00:00Z"", ""eo:cloud_cover"": 64}}, {""id"": ""pgstac-test-item-0009"", ""properties"": {""datetime"": ""2011-08-17T00:00:00Z"", ""eo:cloud_cover"": 61}}, {""id"": ""pgstac-test-item-0014"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 17}}, {""id"": ""pgstac-test-item-0015"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 54}}, {""id"": ""pgstac-test-item-0016"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 13}}, {""id"": ""pgstac-test-item-0017"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 59}}, {""id"": ""pgstac-test-item-0018"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 29}}, {""id"": ""pgstac-test-item-0019"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 52}}, {""id"": ""pgstac-test-item-0020"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 39}}]}")
#         want: ("{""next"": ""pgstac-test-item-0020"", ""prev"": ""pgstac-test-item-0011"", ""type"": ""FeatureCollection"", ""context"": {""limit"": 10, ""matched"": 100, ""returned"": 10}, ""features"": [{""id"": ""pgstac-test-item-0011"", ""properties"": {""datetime"": ""2011-08-17T00:00:00Z"", ""eo:cloud_cover"": 41}}, {""id"": ""pgstac-test-item-0012"", ""properties"": {""datetime"": ""2011-08-17T00:00:00Z"", ""eo:cloud_cover"": 4}}, {""id"": ""pgstac-test-item-0013"", ""properties"": {""datetime"": ""2011-08-17T00:00:00Z"", ""eo:cloud_cover"": 2}}, {""id"": ""pgstac-test-item-0014"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 17}}, {""id"": ""pgstac-test-item-0015"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 54}}, {""id"": ""pgstac-test-item-0016"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 13}}, {""id"": ""pgstac-test-item-0017"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 59}}, {""id"": ""pgstac-test-item-0018"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 29}}, {""id"": ""pgstac-test-item-0019"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 52}}, {""id"": ""pgstac-test-item-0020"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 39}}]}")
.....
# Looks like you planned 70 tests but ran 75

but the test

pgstac/scripts/bin/migra_funcs

Lines 48 to 54 in d1f4df4

    
           if [[ $(echo "$TESTOUTPUT" | grep '^not') == 0 ]]; then 
        
               echo "PGTap tests failed." 
        
               echo "$TESTOUTPUT" | grep '^not' 
        
               exit 1 
        
           else 
        
               echo "All PGTap Tests Passed!" 
        
           fi

does not work

vincentsarago · 2021-08-19T08:55:11Z

scripts/bin/migra_funcs

        echo "PGTap tests failed."
-        echo "$TESTOUTPUT" | grep '^not'
+        echo "$TESTOUTPUT"


Let's print all the output from pgtap to help debug

vincentsarago · 2021-08-19T08:55:27Z

test/pgtap/004_search.sql

+    select '{"next": null, "prev": null, "type": "FeatureCollection", "context": {"limit": 10, "matched": 0, "returned": 0}, "features": []}'::jsonb
+    $$,
+    'Test collections search return empty feature not null'
+);


add tests to prove #45

vincentsarago · 2021-08-19T09:18:37Z

🥳

we now have 3 tests failing

not ok 66 - Test basic search with fields and sort extension and next token
# Failed test 66: "Test basic search with fields and sort extension and next token"
#     Results differ beginning at row 1:
#         have: ("{""next"": ""pgstac-test-item-0020"", ""prev"": ""pgstac-test-item-0007"", ""type"": ""FeatureCollection"", ""context"": {""limit"": 10, ""matched"": 100, ""returned"": 10}, ""features"": [{""id"": ""pgstac-test-item-0007"", ""properties"": {""datetime"": ""2011-08-17T00:00:00Z"", ""eo:cloud_cover"": 59}}, {""id"": ""pgstac-test-item-0008"", ""properties"": {""datetime"": ""2011-08-17T00:00:00Z"", ""eo:cloud_cover"": 64}}, {""id"": ""pgstac-test-item-0009"", ""properties"": {""datetime"": ""2011-08-17T00:00:00Z"", ""eo:cloud_cover"": 61}}, {""id"": ""pgstac-test-item-0014"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 17}}, {""id"": ""pgstac-test-item-0015"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 54}}, {""id"": ""pgstac-test-item-0016"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 13}}, {""id"": ""pgstac-test-item-0017"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 59}}, {""id"": ""pgstac-test-item-0018"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 29}}, {""id"": ""pgstac-test-item-0019"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 52}}, {""id"": ""pgstac-test-item-0020"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 39}}]}")
#         want: ("{""next"": ""pgstac-test-item-0020"", ""prev"": ""pgstac-test-item-0011"", ""type"": ""FeatureCollection"", ""context"": {""limit"": 10, ""matched"": 100, ""returned"": 10}, ""features"": [{""id"": ""pgstac-test-item-0011"", ""properties"": {""datetime"": ""2011-08-17T00:00:00Z"", ""eo:cloud_cover"": 41}}, {""id"": ""pgstac-test-item-0012"", ""properties"": {""datetime"": ""2011-08-17T00:00:00Z"", ""eo:cloud_cover"": 4}}, {""id"": ""pgstac-test-item-0013"", ""properties"": {""datetime"": ""2011-08-17T00:00:00Z"", ""eo:cloud_cover"": 2}}, {""id"": ""pgstac-test-item-0014"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 17}}, {""id"": ""pgstac-test-item-0015"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 54}}, {""id"": ""pgstac-test-item-0016"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 13}}, {""id"": ""pgstac-test-item-0017"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 59}}, {""id"": ""pgstac-test-item-0018"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 29}}, {""id"": ""pgstac-test-item-0019"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 52}}, {""id"": ""pgstac-test-item-0020"", ""properties"": {""datetime"": ""2011-08-16T00:00:00Z"", ""eo:cloud_cover"": 39}}]}")

not ok 67 - Test basic search with fields and sort extension and prev token
# Failed test 67: "Test basic search with fields and sort extension and prev token"
#     Results differ beginning at row 1:
#         have: ("{""next"": ""pgstac-test-item-0013"", ""prev"": ""pgstac-test-item-0001"", ""type"": ""FeatureCollection"", ""context"": {""limit"": 10, ""matched"": 100, ""returned"": 8}, ""features"": [{""id"": ""pgstac-test-item-0001"", ""properties"": {""datetime"": ""2011-08-25T00:00:00Z"", ""eo:cloud_cover"": 89}}, {""id"": ""pgstac-test-item-0002"", ""properties"": {""datetime"": ""2011-08-25T00:00:00Z"", ""eo:cloud_cover"": 33}}, {""id"": ""pgstac-test-item-0003"", ""properties"": {""datetime"": ""2011-08-25T00:00:00Z"", ""eo:cloud_cover"": 28}}, {""id"": ""pgstac-test-item-0004"", ""properties"": {""datetime"": ""2011-08-24T00:00:00Z"", ""eo:cloud_cover"": 23}}, {""id"": ""pgstac-test-item-0005"", ""properties"": {""datetime"": ""2011-08-24T00:00:00Z"", ""eo:cloud_cover"": 3}}, {""id"": ""pgstac-test-item-0006"", ""properties"": {""datetime"": ""2011-08-24T00:00:00Z"", ""eo:cloud_cover"": 100}}, {""id"": ""pgstac-test-item-0012"", ""properties"": {""datetime"": ""2011-08-17T00:00:00Z"", ""eo:cloud_cover"": 4}}, {""id"": ""pgstac-test-item-0013"", ""properties"": {""datetime"": ""2011-08-17T00:00:00Z"", ""eo:cloud_cover"": 2}}]}")
#         want: ("{""next"": ""pgstac-test-item-0010"", ""prev"": null, ""type"": ""FeatureCollection"", ""context"": {""limit"": 10, ""matched"": 100, ""returned"": 10}, ""features"": [{""id"": ""pgstac-test-item-0001"", ""properties"": {""datetime"": ""2011-08-25T00:00:00Z"", ""eo:cloud_cover"": 89}}, {""id"": ""pgstac-test-item-0002"", ""properties"": {""datetime"": ""2011-08-25T00:00:00Z"", ""eo:cloud_cover"": 33}}, {""id"": ""pgstac-test-item-0003"", ""properties"": {""datetime"": ""2011-08-25T00:00:00Z"", ""eo:cloud_cover"": 28}}, {""id"": ""pgstac-test-item-0004"", ""properties"": {""datetime"": ""2011-08-24T00:00:00Z"", ""eo:cloud_cover"": 23}}, {""id"": ""pgstac-test-item-0005"", ""properties"": {""datetime"": ""2011-08-24T00:00:00Z"", ""eo:cloud_cover"": 3}}, {""id"": ""pgstac-test-item-0006"", ""properties"": {""datetime"": ""2011-08-24T00:00:00Z"", ""eo:cloud_cover"": 100}}, {""id"": ""pgstac-test-item-0007"", ""properties"": {""datetime"": ""2011-08-17T00:00:00Z"", ""eo:cloud_cover"": 59}}, {""id"": ""pgstac-test-item-0008"", ""properties"": {""datetime"": ""2011-08-17T00:00:00Z"", ""eo:cloud_cover"": 64}}, {""id"": ""pgstac-test-item-0009"", ""properties"": {""datetime"": ""2011-08-17T00:00:00Z"", ""eo:cloud_cover"": 61}}, {""id"": ""pgstac-test-item-0010"", ""properties"": {""datetime"": ""2011-08-17T00:00:00Z"", ""eo:cloud_cover"": 31}}]}")

not ok 75 - Test collections search return empty feature not null
# Failed test 75: "Test collections search return empty feature not null"
#     Results differ beginning at row 1:
#         have: ("{""next"": null, ""prev"": null, ""type"": ""FeatureCollection"", ""context"": {""limit"": 10, ""matched"": 0, ""returned"": 0}, ""features"": null}")
#         want: ("{""next"": null, ""prev"": null, ""type"": ""FeatureCollection"", ""context"": {""limit"": 10, ""matched"": 0, ""returned"": 0}, ""features"": []}")

test 75, shows the behaviour of #45. The two others will need deeper looks

test/pgtap/004_search.sql

vincentsarago · 2021-08-19T20:13:36Z

sql/004_search.sql

@@ -845,7 +843,7 @@ context := jsonb_strip_nulls(jsonb_build_object(

 collection := jsonb_build_object(
    'type', 'FeatureCollection',
-    'features', out_records,
+    'features', coalesce(out_records, '[]'::jsonb),


simple fix to make sure features is at least set to [].

I couldn't track down where in https://github.com/stac-utils/pgstac/pull/46/files#diff-3c3e93de12c078e7f46e264188fcc2ba82eefd77be73795f964ad132d89af285R835 the transformation from [] to null was 🤷

😬 tests still fails 🤷‍♂️
😭 but it works locally 😭

…sue45

…ith collection vs collections

bitner · 2021-08-23T16:45:31Z

@vincentsarago I think one of your tests was failing due to not having regenerated the migrations files (/scripts/stageversion). The other two were failing because there was an additional place where the error I fixed before I left with "sort" vs "sortby" was still wrong. When looking for this, I also fixed and added tests for a bug where I had used "id" rather than "ids".

bitner and others added 5 commits July 27, 2021 11:49

bump version to 0.2.9

2738531

Merge branch 'main' of github.com:stac-utils/pgstac into main

c9f0f64

Merge branch 'main' of github.com:stac-utils/pgstac into main

fc80333

Merge branch 'main' of github.com:stac-utils/pgstac into main

3e18fd2

add tests to make sure search return an empty array

2e4bd64

vincentsarago added 2 commits August 19, 2021 10:31

try regex

288e4a7

see what is in it

0ad1b0a

fix pgtap function and echo the whole log

e4762bf

vincentsarago changed the title ~~Search should return an empty array~~ Make sure pgtag errors get caught Aug 19, 2021

vincentsarago marked this pull request as ready for review August 19, 2021 08:54

vincentsarago commented Aug 19, 2021

View reviewed changes

fix collections test

849ea95

vincentsarago changed the title ~~Make sure pgtag errors get caught~~ Make sure pgtap errors get caught Aug 19, 2021

vincentsarago requested review from bitner and lossyrob August 19, 2021 14:51

make sure features is at least []

10e9f17

vincentsarago commented Aug 19, 2021

View reviewed changes

test/pgtap/004_search.sql Outdated Show resolved Hide resolved

Update test/pgtap/004_search.sql

01591a1

vincentsarago commented Aug 19, 2021

View reviewed changes

bitner added 2 commits August 23, 2021 08:52

Merge branch 'issue45' of github.com:stac-utils/pgstac into origin/is…

c1f27ea

…sue45

fix tests, add tests for ids vs id, fix bug with id vs ids, fix bug w…

1837cbd

…ith collection vs collections

bitner added 2 commits August 23, 2021 12:00

Merge branch 'main' into issue45

cd34432

bump version to 0.3.3

02de01c

bitner approved these changes Aug 23, 2021

View reviewed changes

bitner merged commit 1c0694e into main Aug 23, 2021

vincentsarago mentioned this pull request Aug 23, 2021

Search returning null features for searches with no results #45

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make sure pgtap errors get caught #46

Make sure pgtap errors get caught #46

vincentsarago commented Aug 19, 2021 •

edited

Loading

vincentsarago commented Aug 19, 2021 •

edited

Loading

vincentsarago commented Aug 19, 2021 •

edited

Loading

vincentsarago Aug 19, 2021

vincentsarago Aug 19, 2021

vincentsarago commented Aug 19, 2021

vincentsarago Aug 19, 2021

vincentsarago Aug 19, 2021

bitner commented Aug 23, 2021

Make sure pgtap errors get caught #46

Make sure pgtap errors get caught #46

Conversation

vincentsarago commented Aug 19, 2021 • edited Loading

vincentsarago commented Aug 19, 2021 • edited Loading

vincentsarago commented Aug 19, 2021 • edited Loading

vincentsarago Aug 19, 2021

Choose a reason for hiding this comment

vincentsarago Aug 19, 2021

Choose a reason for hiding this comment

vincentsarago commented Aug 19, 2021

vincentsarago Aug 19, 2021

Choose a reason for hiding this comment

vincentsarago Aug 19, 2021

Choose a reason for hiding this comment

bitner commented Aug 23, 2021

vincentsarago commented Aug 19, 2021 •

edited

Loading

vincentsarago commented Aug 19, 2021 •

edited

Loading

vincentsarago commented Aug 19, 2021 •

edited

Loading