Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run tests with pytest #963

Merged
merged 48 commits into from
Dec 11, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
38d3e28
Ditch testing for shlex/subprocess modules
craigds Oct 2, 2018
caa6a31
WIP: refactor some dynamic tests to use pytest parametrization
craigds Oct 2, 2018
4452a3b
Pytest config
craigds Oct 2, 2018
9151d45
Remove hacky method to run scripts as imported
craigds Oct 2, 2018
bb0f12e
Minimal changes to fix test collection-time errors with pytest
craigds Oct 2, 2018
064263b
Remove extra sys.path entries when running tests
craigds Oct 2, 2018
2508d81
Refactor more of the dynamic tests to use parametrization
craigds Oct 3, 2018
9d6055e
Refactor leaked FD checks as pytest fixtures
craigds Oct 3, 2018
b3c4596
Add a marker for drivers in tests
craigds Oct 3, 2018
0ad0018
Fix skipif bug
craigds Oct 4, 2018
75c0dc3
Use pytest in CI
craigds Oct 4, 2018
1232bb6
Remove double-parenthesized things in gdaltest_list.append calls
craigds Oct 5, 2018
9b42234
Use pytest parametrize for dynamically generated tests
craigds Oct 5, 2018
e45ca89
Manual fixes for a strange syntax error in the automation
craigds Oct 5, 2018
06456cd
Simplify gdaltest_list usage
craigds Oct 5, 2018
0c7b2b3
Remove some no-longer-necessary stuff from gdaltest module
craigds Oct 16, 2018
53a3850
Add a script to setup virtualenv
craigds Oct 19, 2018
235004d
Fix weird comparison in tiff_ovr breaking condition inversion
craigds Oct 26, 2018
a56395f
Update `with_and_without_postgis` to a pytest fixture in ogr_pg.py
craigds Oct 29, 2018
19b48ca
Fix builds on OSX
craigds Nov 5, 2018
1cdd430
Manual fixes for wfs/csw tests
craigds Nov 29, 2018
b195f51
Automated: Rename all python tests to test_*
craigds Dec 9, 2018
1a5aa27
Automated: Don't expect 'success' from test helpers
craigds Dec 9, 2018
81a03cb
Automated: Remove useless `post_reason() calls
craigds Dec 9, 2018
3fe4e7d
Automated: 'return fail' --> assert
craigds Dec 9, 2018
18d4163
Automated: Rewrite most skip/fail from tests
craigds Dec 9, 2018
1e0db0c
Automated: Don't return 'success' from tests
craigds Dec 9, 2018
b81466c
Automated: Remove gdaltest_list
craigds Dec 9, 2018
9412c2d
Automated: Remove __main__ block from tests
craigds Dec 9, 2018
2ab1609
Automated: Use pytest.raises() where necessary
craigds Dec 9, 2018
2e4ce80
Automated: Some autoflake cleanup
craigds Dec 9, 2018
3e128be
Assorted manual test failure fixes
craigds Oct 21, 2018
9771c21
Some more autoflake cleanup
craigds Nov 5, 2018
20c4ebd
Remove cleanup_on_exit hook
craigds Nov 7, 2018
d4cacb3
Fix license-checking test
craigds Nov 12, 2018
a57ee5a
Disable a LC_CTYPE test in test_gdal2tiles
craigds Nov 12, 2018
d87dc34
Fix sqlite tests when compiled without spatialite
craigds Nov 14, 2018
ada1eb4
Run tests in per-module subprocesses in the address sanitizer env
craigds Nov 15, 2018
a0d6a07
Remove run_tests() function, remove last usages of it
craigds Nov 17, 2018
b65c5d1
MinGW build tweaks
craigds Nov 20, 2018
7e62d38
Add some basic docs. Remove pin for pytest; v4 should work ok
craigds Nov 29, 2018
33b7a95
Remove a number of gdaltest_list mentions, no longer needed/used
craigds Nov 29, 2018
41e15ac
Update appveyor/windows build to use pytest
craigds Nov 29, 2018
10d7106
Include but skip dods/gmlas tests. Include fgdb stress tests if fgdb …
craigds Nov 29, 2018
6c3d9aa
Add more verbosity to test output in CI
craigds Nov 30, 2018
b54e910
Make vrt_read_29 test not care whether gdal.Translate closes the dest…
craigds Nov 30, 2018
46e92b7
Tweak the IDs in parametrized tests to be more useful
craigds Dec 9, 2018
b2453ed
Fix bug in vagrantfile script to install pip
craigds Dec 11, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
3 changes: 2 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ gdalinfo --version
Run autotest suite:
```
cd ../autotest
python run_all.py
pip install -r requirements.txt
pytest
```

Git workflows with GDAL
Expand Down
3 changes: 2 additions & 1 deletion appveyor.yml
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,8 @@ test_script:
cd $env:APPVEYOR_BUILD_FOLDER\autotest\cpp
exec { nmake /f makefile.vc check MSVC_VER=$env:MSVC_VER $env:WIN64_ARG }
cd $env:APPVEYOR_BUILD_FOLDER\autotest
exec { python run_all.py }
exec { pip install -Ur requirements.txt }
exec { pytest -vv }
cd $env:APPVEYOR_BUILD_FOLDER\gdal\swig\csharp
exec { nmake /f makefile.vc test}

Expand Down
14 changes: 7 additions & 7 deletions autotest/GNUmakefile
Original file line number Diff line number Diff line change
Expand Up @@ -2,25 +2,25 @@ all:
@echo "Run 'make -j test'"

gcore-test:
python run_all.py gcore
pytest gcore

gdrivers-test:
python run_all.py gdrivers
pytest gdrivers

alg-test:
python run_all.py alg
pytest alg

osr-test:
python run_all.py osr
pytest osr

ogr-test:
python run_all.py ogr
pytest ogr

utilities-test:
python run_all.py utilities
pytest utilities

pyscripts-test:
python run_all.py pyscripts
pytest pyscripts

test check:
@$(MAKE) $(MFLAGS) gcore-test gdrivers-test alg-test ogr-test osr-test utilities-test pyscripts-test
15 changes: 0 additions & 15 deletions autotest/README

This file was deleted.

56 changes: 56 additions & 0 deletions autotest/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# GDAL test suite


## How to run tests

1. You need to install `pytest` to run the test suite. This should do it:

```bash
cd autotest
pip install -r ./requirements.txt
```

2. Then, run tests with:

```bash
pytest
```

3. Some quick usage tips:

```bash
# get more verbose output; don't capture stdout/stdin
pytest -vvs

# run all the gcore tests
pytest gcore/

# run a particular module only
pytest gcore/basic_test.py
```


## GDAL's tests are not independent

GDAL's test functions are not currently independent of each other. In particular, running individual test functions from a given module may not work. Most tests were originally written with the assumption that entire modules will be run at once.

Practically, this means that you should avoid using:


* pytest's `--last-failed` / `--lf` option (since it runs only failed tests, not the whole module)
* test specifiers that run individual tests (e.g. `pytest gcore/basic_test.py::test_basic_test_1` )
* the xunit plugin to run tests in parallel, unless you also use `--dist=loadfile`. (This may have other issues; untested)

This will hopefully be addressed in the future. When writing new tests, please try to make them independent of each other.


## Notes about availability of GDAL sample and test data

The GDAL Team makes every effort to assure that all sample data files
available from GDAL download server (http://download.osgeo.org/gdal/data/) and
test data files used in GDAL Autotest package (https://github.com/OSGeo/gdal/tree/master/autotest)
are available as public and freely redistributable geodata.

--

http://gdal.org/
Empty file added autotest/alg/__init__.py
Empty file.
Loading