Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Phase 7] Complete Sharrow #611

Merged
merged 399 commits into from
Jan 3, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
399 commits
Select commit Hold shift + click to select a range
880edac
use unique env file
jpn-- Jun 14, 2022
5722749
install without deps
jpn-- Jun 14, 2022
007eaf2
other core tests
jpn-- Jun 14, 2022
3b57e22
fix sandag tests
jpn-- Jun 15, 2022
b389ad0
add mtc tests to GA
jpn-- Jun 15, 2022
c55bf08
top constrain pytables
jpn-- Jun 15, 2022
aab5529
fix psrc test
jpn-- Jun 15, 2022
b2389b1
sharrowcache
jpn-- Jun 15, 2022
843c959
matrixed example models
jpn-- Jun 16, 2022
65c9b6b
nested hashing
jpn-- Jun 16, 2022
c1cb409
better name
jpn-- Jun 16, 2022
bfa533d
comprehensive regional test matrix
jpn-- Jun 16, 2022
77fdeff
fix workflow
jpn-- Jun 16, 2022
9e78765
don't fail-fast on regional models, depend only on linux foundation
jpn-- Jun 16, 2022
3893bc9
fix GA
jpn-- Jun 16, 2022
cfb2288
fix for multizone on GA
jpn-- Jun 16, 2022
22a0f4f
REGION_DEFS
jpn-- Jun 16, 2022
2c0e0d2
reset cache
jpn-- Jun 17, 2022
f940cf5
repair sandag xborder model
jpn-- Jun 17, 2022
4da3c61
blacken
jpn-- Jun 17, 2022
4c54c9e
fix macos cache
jpn-- Jun 17, 2022
cb15d7f
aggressive caching
jpn-- Jun 17, 2022
d1a212c
unpin numpy
jpn-- Jun 19, 2022
11303ab
test estimation, cache artifacts
jpn-- Jun 20, 2022
a73a746
sandag sharrow testing
jpn-- Jun 21, 2022
ec8d2de
sandag-1zone-mini
jpn-- Jun 22, 2022
85e5059
init chunked workflow
jpn-- Jun 22, 2022
25b46a0
sandag-1zone-complete
jpn-- Jun 22, 2022
876c18a
bugfix
jpn-- Jun 22, 2022
60b1c50
chunked sandag 1
jpn-- Jun 22, 2022
37e8533
warn dont fail
jpn-- Jun 23, 2022
3977f89
workflow updates
jpn-- Jun 23, 2022
773caba
skipable chunk training
jpn-- Jun 24, 2022
c867921
blacken
jpn-- Jun 24, 2022
02311b2
chunk size detection
jpn-- Jul 12, 2022
3bfa43c
use settings_file in compile
jpn-- Jul 12, 2022
4bc5b47
fix recursion
jpn-- Jul 12, 2022
aa21cd0
fix deadlock in reference MP
jpn-- Jul 13, 2022
8b53ca6
blacken
jpn-- Jul 13, 2022
b0fd70a
comprehensive workflow
jpn-- Jul 13, 2022
22b5efa
Ensure integer types for numbers
jpn-- Jul 13, 2022
ec98d75
add memory_profiling to comprehensive
jpn-- Jul 13, 2022
5fadb20
add ARC single thread to comprehensive
jpn-- Jul 13, 2022
4e42bb4
workflow notes
jpn-- Jul 13, 2022
60e06ef
repair integers
jpn-- Jul 14, 2022
5eef3d2
avoid interaction_df in interaction_simulate if no skims or tracing
jpn-- Jul 14, 2022
14ff73e
trim memory peak in time table
jpn-- Jul 14, 2022
b2abd06
forced garbage collection option
jpn-- Jul 14, 2022
024539a
short circuit chunk logic on disabled chunking
jpn-- Jul 15, 2022
fcf905a
swallow errors in workflow
jpn-- Jul 15, 2022
7eada19
settings for mtc_comprehensive
jpn-- Jul 15, 2022
675ede0
fix num_processes
jpn-- Jul 15, 2022
b9e23ff
sandag mp chunked
jpn-- Jul 15, 2022
0ed85e4
sandag create for mp
jpn-- Jul 15, 2022
effc637
workflow name for chunked
jpn-- Jul 15, 2022
c084425
swallow errors in chunked
jpn-- Jul 16, 2022
ea902a4
seperate chunk cache for legacy and sharrow
jpn-- Jul 22, 2022
1969f87
default chunk size should be None
jpn-- Jul 22, 2022
cf687d9
documentation cleanup
jpn-- Jul 22, 2022
4ec5531
build docs
jpn-- Jul 22, 2022
89e5e34
dummy code in setup.py
jpn-- Jul 22, 2022
0f36391
docbuild env
jpn-- Jul 23, 2022
2367ecc
blacken
jpn-- Jul 23, 2022
b3aaf4a
clean up workflow
jpn-- Jul 23, 2022
632d88b
cache separate leg and sh
jpn-- Jul 23, 2022
890861d
fix assertion error when chunking is disabled and adaptive chunking i…
jpn-- Jul 23, 2022
d048ead
blacken
jpn-- Jul 23, 2022
17ea961
repair fallback version
jpn-- Jul 25, 2022
e819d07
sandag 1zone mp workflow
jpn-- Jul 25, 2022
8239d1c
recode TAZs also
jpn-- Jul 26, 2022
36fc577
select zone layer in flows
jpn-- Jul 26, 2022
998e231
arc_mp
jpn-- Jul 27, 2022
04e9e54
Merge branch 'with-sharrow' into with-sharrow-2z
jpn-- Jul 27, 2022
e8fcb12
fix cols on ARC
jpn-- Jul 27, 2022
61923c6
use chunk training
jpn-- Jul 27, 2022
f59c8bb
trace in compile
jpn-- Jul 27, 2022
7a42884
maz and taz in flows
jpn-- Jul 28, 2022
89b9d47
two zone complete run
jpn-- Jul 29, 2022
ba74c44
refactor skim_dataset
jpn-- Jul 30, 2022
107e270
turn on chunk training by default
jpn-- Aug 1, 2022
8fc3041
Merge branch 'develop' into sharrow-black
jpn-- Aug 6, 2022
618413d
fix merge error
jpn-- Aug 6, 2022
4f12df4
change linter to Black
jpn-- Aug 6, 2022
86850f2
merge error, lost timing notes
jpn-- Aug 6, 2022
77ea21e
merge error in testing
jpn-- Aug 6, 2022
be4de1f
install black in Github Actions
jpn-- Aug 6, 2022
7d1da5f
black and isort again
jpn-- Aug 6, 2022
f61db75
ignore _generated_version.py
jpn-- Aug 6, 2022
f5fa6c7
testing cleanup
jpn-- Aug 6, 2022
ec9a50b
pre-commit hooks
jpn-- Aug 6, 2022
90e659c
blacken, flake8 config
jpn-- Aug 6, 2022
4fb702c
fix regional test names for new scheme
jpn-- Aug 6, 2022
c52496f
clean naming
jpn-- Aug 6, 2022
d0cff5d
black linting on crossplatform
jpn-- Aug 6, 2022
da2172f
re-enable sharrow testing
jpn-- Aug 6, 2022
bd326f9
allow unused vars
jpn-- Aug 7, 2022
6720179
fix merge problems in proto mtc
jpn-- Aug 7, 2022
8784e35
Merge commit 'e92ac727454b1f1027fb7146a1ef130be1509580' into sharrow-…
jpn-- Aug 8, 2022
844eec2
remove run_flags
jpn-- Aug 8, 2022
dc9d7f5
settings docs
jpn-- Aug 9, 2022
633a362
add new settings to docs
jpn-- Aug 9, 2022
345fe8a
docs update for publishing on CS fork
jpn-- Aug 9, 2022
ed8dcff
update docs to ref black not pycodestyle
jpn-- Aug 9, 2022
5b60411
Merge branch 'sharrow-black' into with-sharrow-2z
jpn-- Aug 10, 2022
ce484a2
repair workflows
jpn-- Aug 10, 2022
56aedce
fix black version
jpn-- Aug 10, 2022
83c0b15
repair ARC MP
jpn-- Aug 10, 2022
c853d9f
contrast distance on 2 zone models
jpn-- Aug 10, 2022
04ae559
zarr with 2 zone
jpn-- Aug 10, 2022
ffbd9fe
sandag 2 zone sharrow tests
jpn-- Aug 10, 2022
cd0d617
land_use_taz table is needed for 2 zone reconciliation
jpn-- Aug 10, 2022
055e1d5
sandag test fixes
jpn-- Aug 11, 2022
9ff3c8f
Merge branch 'master' of https://github.com/ActivitySim/activitysim i…
jpn-- Aug 12, 2022
4038294
chunk training on less-than-full population
jpn-- Aug 12, 2022
01c729d
sandag 3 settings
jpn-- Aug 12, 2022
c073fca
limit to setuptools<64
jpn-- Aug 12, 2022
da18859
3 zones running
jpn-- Aug 16, 2022
c548dee
handle when time period is already the i
jpn-- Aug 16, 2022
55556bf
_preprocess_tvpb_logsums_on_choosers
jpn-- Aug 16, 2022
107ba2e
repair sandag 3 model
jpn-- Aug 16, 2022
b9df027
use SkimDict in legacy
jpn-- Aug 17, 2022
19d8e36
school and work dist in report
jpn-- Aug 17, 2022
6058c0a
chunk training on only 2 processes
jpn-- Aug 17, 2022
2e73a39
fix accessibility
jpn-- Aug 18, 2022
16062d6
3 zone up
jpn-- Aug 20, 2022
600098f
3 zone ref
jpn-- Aug 22, 2022
03bbb7c
sandag xborder sharrow
jpn-- Aug 22, 2022
0862e84
exts
jpn-- Aug 23, 2022
8250e84
bleed docs on 3z
jpn-- Aug 23, 2022
ef1c5c2
use latest publisher
jpn-- Aug 23, 2022
78bfa37
do nothing if table missing
jpn-- Aug 23, 2022
b0fdc7a
fix report (partial)
jpn-- Aug 23, 2022
c9610f3
Merge commit '3e254ff6dafd15b55a4eac0ac7c4bffe4eab93b7' into sharrow-…
jpn-- Aug 23, 2022
1d56952
Merge commit '30455f289e2ea46196501d4f0625ebe29047bd4a' into sharrow-3z
jpn-- Aug 23, 2022
a875ba5
fix bug
jpn-- Aug 24, 2022
2a92be8
Merge commit 'b61f78bfa0a8ebf725052719349737708e5ed863' into sharrow-3z
jpn-- Aug 24, 2022
c068dff
drop redundant workflow
jpn-- Aug 24, 2022
2364656
Merge commit 'c6b9c9f40f1a85aafdd688b9358023a02a6588ec' into sharrow-3z
jpn-- Aug 24, 2022
3ceb324
Merge commit '9a782f8f379c71b88df34443b2426daa9b653a71' into sharrow-3z
jpn-- Aug 24, 2022
21f3ff5
Merge commit '8b7737cec695c0da74bcea694617793d0104f9a4' into feature-…
jpn-- Aug 24, 2022
e49ad08
multiprocess admonition [makedocs]
jpn-- Aug 24, 2022
289e9b6
allow missing land_use_taz
jpn-- Aug 24, 2022
a6ceffc
repair 2 zone test data [makedocs]
jpn-- Aug 25, 2022
5b736f6
fix style in docs [makedocs]
jpn-- Aug 25, 2022
1d7fbbf
doc disable_zarr
jpn-- Aug 25, 2022
4714d95
don't cache logsums
jpn-- Aug 25, 2022
03e8468
sort dependencies
jpn-- Aug 26, 2022
5f47257
fix merge error [makedocs]
jpn-- Aug 26, 2022
fc7aa14
test fixes
jpn-- Aug 26, 2022
a6bc910
Merge branch 'develop' into feature-sharrow-3z
jpn-- Sep 2, 2022
f89b56d
sharrow testing 3zone
jpn-- Sep 2, 2022
1dec08c
test fixing
jpn-- Sep 2, 2022
4c251c3
fix pipeline recoding in multizone tests
jpn-- Sep 2, 2022
0ba8db3
custom_step can overload existing steps
jpn-- Sep 6, 2022
5a5c39a
fix logsum caching in 3-zone
jpn-- Sep 6, 2022
d9561d8
no simulate skipping based on component name
jpn-- Sep 7, 2022
d27d813
sharrow tt works
jpn-- Sep 9, 2022
67fafa3
speed up nonmand schedule
jpn-- Sep 11, 2022
73efdaf
add docs
jpn-- Sep 12, 2022
017b87e
protect memory profile
jpn-- Sep 12, 2022
464ef26
series begets series
jpn-- Sep 13, 2022
5f6412a
merging in develop
dhensle Sep 14, 2022
4ae569a
blacken
dhensle Sep 14, 2022
5027b4a
mwcog fixes
jpn-- Sep 14, 2022
65240d0
mwcog sharrow tests
jpn-- Sep 15, 2022
f6f7027
repair ARC
jpn-- Sep 15, 2022
fd044bb
update dependencies
jpn-- Sep 16, 2022
0e3f87c
Update trip_destination.py
dhensle Sep 16, 2022
b7e08c4
Update trip_destination.py
dhensle Sep 16, 2022
ee9603f
fix --ext
jpn-- Sep 17, 2022
08d699a
repair sandag_xborder test
jpn-- Sep 17, 2022
4171f18
sandag xborder full tests
jpn-- Sep 17, 2022
c0d0e68
repair tests for 2 zone
jpn-- Sep 17, 2022
0849d34
sandag test fix
jpn-- Sep 17, 2022
b29ee87
fix sandag tests
jpn-- Sep 18, 2022
1768271
revise est tests
jpn-- Sep 18, 2022
2f6de1a
estimation test updates
jpn-- Sep 18, 2022
17d6ec9
sharrow for mtc_extended
jpn-- Sep 19, 2022
4fbb0fd
docs
jpn-- Sep 21, 2022
e6257ac
bump sharrow
jpn-- Sep 21, 2022
1f43fec
docs
jpn-- Sep 21, 2022
d9c58e0
drop skim_dataset from aggregate_size_terms
jpn-- Sep 21, 2022
34a2a23
pandas < 1.5 for now
jpn-- Sep 22, 2022
be3b1a8
workflow_ silently ignores missing workflows in tests
jpn-- Sep 23, 2022
e30bcba
doc updates
jpn-- Sep 23, 2022
65669b9
spec repairs
jpn-- Sep 23, 2022
a5ee05c
psrc test repair
jpn-- Sep 23, 2022
8ead50e
test repairs
jpn-- Sep 23, 2022
f22fd15
recode_pipeline_columns for sh in workflows
jpn-- Sep 23, 2022
24bc1ee
mwcog needs ext
jpn-- Sep 23, 2022
0cf625c
exercise-mini
jpn-- Sep 24, 2022
b4cd888
sharrow_skip
jpn-- Sep 24, 2022
2367df6
failable sharrow_skip
jpn-- Sep 24, 2022
f41160c
better comprehensive
jpn-- Sep 24, 2022
e6d140c
reduce default arc_full workflow loading
jpn-- Sep 24, 2022
73ec6b6
comprehensive-mini
jpn-- Sep 25, 2022
ef44ef6
use 1.1.3 as ref
jpn-- Sep 25, 2022
091651b
repair arc and psrc tests
jpn-- Sep 26, 2022
f40a073
repair sandag xborder
jpn-- Sep 26, 2022
782eee0
repair sandag2 tests
jpn-- Sep 26, 2022
e545e99
trip dest docs
jpn-- Sep 27, 2022
0d53eca
[makedocs]
jpn-- Sep 27, 2022
b4dd8b2
reduce duplication
jpn-- Sep 27, 2022
3a3d6a6
more docs
jpn-- Sep 27, 2022
7506dbc
trip_destination_annotate_trips_preprocessor for sandag 1 zone in 1.1.3
jpn-- Sep 27, 2022
d9cd6e5
how to turn off digital encoding [makedocs]
jpn-- Sep 28, 2022
ec285e5
error here, not just empty dict. should never see it
jpn-- Sep 28, 2022
c4fea70
more doc [makedocs]
jpn-- Sep 29, 2022
658b936
generalized digital encoding
jpn-- Sep 29, 2022
a58b691
fix error [makedocs]
jpn-- Sep 29, 2022
8d69fad
numba 56 not working yet
jpn-- Sep 29, 2022
19fe95e
doc updates [makedocs]
jpn-- Sep 29, 2022
26f8b12
minor repairs
jpn-- Oct 3, 2022
3770e5a
skip when household_income is None
jpn-- Oct 3, 2022
290c43c
add mwcog full example
jpn-- Nov 22, 2022
f9483e0
warn don't fail on duplicate skims
jpn-- Dec 11, 2022
aa1aaa4
include sharrow in mwcog example
jpn-- Dec 11, 2022
faa7a39
cleanup
jpn-- Dec 11, 2022
97225b3
zarr digital encoding for mwcog
jpn-- Dec 11, 2022
88dc1cf
include reference in psrc mini
jpn-- Dec 11, 2022
80812d0
fow window ids to int64 because windows
jpn-- Dec 12, 2022
dbe0da9
Merge branch 'bugfix-warnings' into feature-sharrow-tt
jpn-- Dec 12, 2022
459bdb1
columns cannot be a set
jpn-- Dec 12, 2022
6943d64
Merge branch 'develop' into feature-sharrow-tt
jpn-- Dec 12, 2022
7c03020
change 'arry' to more readable 'sizearray'
jpn-- Dec 12, 2022
6c79abc
repair tests
jpn-- Dec 12, 2022
4cb65ca
catch FileNotFound
jpn-- Dec 13, 2022
231e7a3
add check that land use is zero-based when using sharrow
jpn-- Dec 13, 2022
33deb63
better docs on skims in shared memory
jpn-- Dec 13, 2022
3b132e7
better docstrings for skimdataset
jpn-- Dec 13, 2022
c6e68af
remove old code
jpn-- Dec 13, 2022
a8ab2ff
remove junk
jpn-- Dec 13, 2022
da51038
add logging for recoding
jpn-- Dec 13, 2022
be0319c
Merge branch 'develop' into feature-sharrow-tt
jpn-- Dec 21, 2022
e270830
blacken
jpn-- Dec 22, 2022
f6ee1e9
Merge branch 'develop_tripDestUpdate' into feature-sharrow-tt
jpn-- Dec 22, 2022
3a4f1e8
fix merge error
jpn-- Dec 22, 2022
1039c90
allow pandas 1.5 again
jpn-- Dec 22, 2022
5e61a2c
sharrow compatibility with #606
jpn-- Dec 22, 2022
02720b3
remove outdated code in comments
jpn-- Dec 22, 2022
98034cc
note on tracing limitation
jpn-- Dec 22, 2022
76125a8
allow repeating key cols
jpn-- Dec 22, 2022
0295619
fix decoding in MP output
jpn-- Dec 22, 2022
bcc89c8
Update .gitignore
jpn-- Jan 2, 2023
6194319
Merge branch 'develop' into feature-sharrow-tt
jpn-- Jan 2, 2023
d56bef6
ignore caches
jpn-- Jan 2, 2023
8dca8e9
remove special install instructions
jpn-- Jan 2, 2023
6b52af2
remove sharrow_skip's
jpn-- Jan 2, 2023
ea24043
Revert "remove sharrow_skip's"
jpn-- Jan 2, 2023
60f8dda
add note about why sharrow_skip is activated
jpn-- Jan 2, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
2 changes: 2 additions & 0 deletions .github/workflows/core_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@ jobs:
run: |
python -m pytest --pyargs activitysim.cli


cross-platform:
# also test foundation cross platforms, but do not require a successful
# completion before starting regional model tests
Expand Down Expand Up @@ -162,6 +163,7 @@ jobs:
run: |
python -m pytest --pyargs activitysim.cli


regional_models:
needs: foundation
env:
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,8 @@ _test_est
*_local/
*_local.*

**/__sharrowcache__
jpn-- marked this conversation as resolved.
Show resolved Hide resolved
**/skims.zarr
**/output/
**/_generated_version.py
docs/**/_generated
2 changes: 0 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,6 @@ repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.3.0
hooks:
- id: check-yaml
exclude: logging.yaml # TODO don't exclude, will require fixing logging
- id: end-of-file-fixer
exclude: .*\.ipynb
- id: trailing-whitespace
Expand Down
88 changes: 44 additions & 44 deletions HOW_TO_RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@
01. Check that the branch you intend to release is passing tests on Travis.
If it's not passing there you should not release it.

00. Start from a completely clean conda environment
and git repository. Assuming you have `conda` installed, you can do so
by starting where ActivitySim is not yet cloned (e.g. in an empty
00. Start from a completely clean conda environment
and git repository. Assuming you have `conda` installed, you can do so
by starting where ActivitySim is not yet cloned (e.g. in an empty
directory) and running:
```sh
conda create -n TEMP-ASIM-DEV python=3.9 git gh -c conda-forge --override-channels
Expand All @@ -19,7 +19,7 @@

00. Per project policy, code on the main branch should have been released,
but if you are *preparing* a release then the code should be on the `develop`
branch. Switch to that branch now, and make sure it is synced to the
branch. Switch to that branch now, and make sure it is synced to the
version on GitHub:
```sh
git switch develop
Expand All @@ -28,45 +28,45 @@

00. Update your Conda environment for testing. We do not want to use an
existing environment on your machine, as it may be out-of-date
and we want to make sure everything passes muster using the
and we want to make sure everything passes muster using the
most up-to-date dependencies available. The following command
will update the active environment (we made this to be `TEMP-ASIM-DEV`
will update the active environment (we made this to be `TEMP-ASIM-DEV`
if you followed the directions above).
```sh
conda env update --file=conda-environments/activitysim-dev.yml
```
If you add to the ActivitySim dependencies, make sure to also update
the environments in `conda-environments`, which are used for testing
and development. If they are not updated, these environments will end
If you add to the ActivitySim dependencies, make sure to also update
the environments in `conda-environments`, which are used for testing
and development. If they are not updated, these environments will end
up with dependencies loaded from *pip* instead of *conda-forge*.

00. Run pycodestyle to ensure that the codebase passes all style checks.
00. Run black to ensure that the codebase passes all style checks.
This check should only take a few seconds. These checks are also done on
Travis and are platform independent, so they should not be necessary to
replicate locally, but are listed here for completeness.
```sh
pycodestyle .
black --check --diff .
```

00. Run the regular test suite on Windows. Travis tests are done on Linux,
but most users are on Windows, and the test suite should also be run
on Windows to ensure that it works on that platform as well. If you
are not preparing this release on Windows, you should be sure to run
at least through this step on a Windows machine before finalizing a
release.
A few of the tests require pre-created data that is not included in the
repository directly, but rather recreated on the fly before testing. The
regular test suite takes some time to run, between about half an hour and
at least through this step on a Windows machine before finalizing a
release.

A few of the tests require pre-created data that is not included in the
repository directly, but rather recreated on the fly before testing. The
regular test suite takes some time to run, between about half an hour and
two hours depending on the specs of your machine.
```sh
python activitysim/examples/placeholder_multiple_zone/scripts/two_zone_example_data.py
python activitysim/examples/placeholder_multiple_zone/scripts/three_zone_example_data.py
pytest .
```

00. Test the full-scale regional examples. These examples are big, too
large to run on Travis, and will take a lot of time (many hours) to
large to run on Travis, and will take a lot of time (many hours) to
download and run.
```sh
mkdir tmp-asim
Expand All @@ -76,39 +76,39 @@
call run_all_examples.bat
```
These tests will run through the gamut even if some of them crash, so
if you don't sit and watch them go (please don't do this) you'll need
if you don't sit and watch them go (please don't do this) you'll need
to scan through the results to make sure there are no errors after the
fact.
```sh
python ../activitysim/examples/scan_examples_for_errors.py .
```

00. Test the notebooks in `activitysim/examples/prototype_mtc/notebooks`.
There are also demo notebooks for estimation, but their functionality
There are also demo notebooks for estimation, but their functionality
is completely tested in the unit tests run previously.

00. Use bump2version to tag the release commit and update the
00. Use bump2version to tag the release commit and update the
version number. The following code will generate a "patch" release,
incrementing the third value in the version number (i.e. "1.2.3"
becomes "1.2.4"). Alternatively, make a "minor" or "major" release.
The `--list` command will generate output to your console to confirm
that the old and new version numbers are what you expect, before you
push the commit (with the changed version in the code) and tags to
incrementing the third value in the version number (i.e. "1.2.3"
becomes "1.2.4"). Alternatively, make a "minor" or "major" release.
The `--list` command will generate output to your console to confirm
that the old and new version numbers are what you expect, before you
push the commit (with the changed version in the code) and tags to
GitHub.
```sh
bump2version patch --list
```
It is also possible to make a development pre-release. To do so,
explicitly set the version number to the next patch plus a ".devN"
suffix:

It is also possible to make a development pre-release. To do so,
explicitly set the version number to the next patch plus a ".devN"
suffix:

```sh
bump2version patch --new-version 1.2.3.dev0 --list
```
Then, when ready to make a "final" release, set the version by
explicitly removing the suffix:

Then, when ready to make a "final" release, set the version by
explicitly removing the suffix:
```sh
bump2version patch --new-version 1.2.3 --list
```
Expand All @@ -118,26 +118,26 @@
git push --tags
```

00. For non-development releases, open a pull request to merge the proposed
release into main. The following command will open a web browser for
00. For non-development releases, open a pull request to merge the proposed
release into main. The following command will open a web browser for
you to create the pull request.
```sh
gh pr create --web
```
After creating the PR, confirm with the ActivitySim PMC that the release
is ready before actually merging it.

Once final approval is granted, merge the PR into main. The presence
of the git tags added earlier will trigger automated build steps to
prepare and deploy the release to pypi and conda-forge.

00. Create a "release" on GitHub.
```sh
gh release create v1.2.3
```
For a development pre-release, include the `--prerelease` argument.
As the project's policy is that only formally released code is merged
to the main branch, any pre-release should also be built against a
to the main branch, any pre-release should also be built against a
non-default branch. For example, to pre-release from the `develop`
branch:
```sh
Expand All @@ -147,11 +147,11 @@
--notes "this pre-release is for a cool new feature" \
--title "Development Pre-Release"
```
00. Clean up your workspace, including removing the Conda environment used for
testing (which will prevent you from accidentally using an old

00. Clean up your workspace, including removing the Conda environment used for
testing (which will prevent you from accidentally using an old
environment when you should have a fresh up-to-date one next time).
```sh
conda deactivate
conda env remove -n TEMP-ASIM-DEV
```
```
4 changes: 4 additions & 0 deletions activitysim/abm/models/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,12 @@
parking_location_choice,
stop_frequency,
summarize,
telecommute_frequency,
tour_mode_choice,
tour_od_choice,
tour_scheduling_probabilistic,
transit_pass_ownership,
transit_pass_subsidy,
trip_departure_choice,
trip_destination,
trip_matrices,
Expand All @@ -39,4 +42,5 @@
trip_scheduling_choice,
vehicle_allocation,
vehicle_type_choice,
work_from_home,
)
2 changes: 2 additions & 0 deletions activitysim/abm/models/atwork_subtour_scheduling.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ def atwork_subtour_scheduling(
estimator = estimation.manager.begin_estimation("atwork_subtour_scheduling")

model_spec = simulate.read_model_spec(file_name=model_settings["SPEC"])
sharrow_skip = model_settings.get("sharrow_skip")
coefficients_df = simulate.read_model_coefficients(model_settings)
model_spec = simulate.eval_coefficients(model_spec, coefficients_df, estimator)

Expand Down Expand Up @@ -79,6 +80,7 @@ def atwork_subtour_scheduling(
estimator=estimator,
chunk_size=chunk_size,
trace_label=trace_label,
sharrow_skip=sharrow_skip,
)

if estimator:
Expand Down
12 changes: 10 additions & 2 deletions activitysim/abm/models/initialize_los.py
Original file line number Diff line number Diff line change
Expand Up @@ -154,8 +154,16 @@ def compute_utilities_for_attribute_tuple(
choosers_df, chunk_size, trace_label, chunk_tag=chunk_tag
):
# we should count choosers_df as chunk overhead since its pretty big and was custom made for compute_utilities
assert chooser_chunk._is_view # otherwise copying it is wasteful
chooser_chunk = chooser_chunk.copy()
if chooser_chunk._is_view:
chooser_chunk = (
chooser_chunk.copy()
) # copy is needed when we start with a view
else:
# copying this is wasteful, but the code below edits the dataframe,
# which could have undesirable side effects.
# TODO: convert to Dataset or otherwise stop this copying, without
# harming anything else.
chooser_chunk = chooser_chunk.copy()
chunk.log_df(trace_label, "attribute_chooser_chunk", chooser_chunk)

# add any attribute columns specified as column attributes in settings (the rest will be scalars in locals_dict)
Expand Down
2 changes: 2 additions & 0 deletions activitysim/abm/models/joint_tour_participation.py
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,8 @@ def participants_chooser(probs, choosers, spec, trace_label):
In principal, this shold always occur eventually, but we fail after MAX_ITERATIONS,
just in case there is some failure in program logic (haven't seen this occur.)

The return values are the same as logit.make_choices

Parameters
----------
probs : pandas.DataFrame
Expand Down
2 changes: 2 additions & 0 deletions activitysim/abm/models/joint_tour_scheduling.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ def joint_tour_scheduling(tours, persons_merged, tdd_alts, chunk_size, trace_hh_
estimator = estimation.manager.begin_estimation("joint_tour_scheduling")

model_spec = simulate.read_model_spec(file_name=model_settings["SPEC"])
sharrow_skip = model_settings.get("sharrow_skip", False)
coefficients_df = simulate.read_model_coefficients(model_settings)
model_spec = simulate.eval_coefficients(model_spec, coefficients_df, estimator)

Expand All @@ -91,6 +92,7 @@ def joint_tour_scheduling(tours, persons_merged, tdd_alts, chunk_size, trace_hh_
estimator=estimator,
chunk_size=chunk_size,
trace_label=trace_label,
sharrow_skip=sharrow_skip,
)

if estimator:
Expand Down
35 changes: 21 additions & 14 deletions activitysim/abm/models/location_choice.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,7 @@
config,
expressions,
inject,
logit,
los,
mem,
pipeline,
simulate,
tracing,
Expand Down Expand Up @@ -116,6 +114,7 @@ def _location_sample(
chunk_size,
chunk_tag,
trace_label,
zone_layer=None,
):
"""
select a sample of alternative locations.
Expand Down Expand Up @@ -149,7 +148,13 @@ def _location_sample(
)
sample_size = 0

locals_d = {"skims": skims, "segment_size": segment_name}
locals_d = {
"skims": skims,
"segment_size": segment_name,
"orig_col_name": skims.orig_key, # added for sharrow flows
"dest_col_name": skims.dest_key, # added for sharrow flows
"timeframe": "timeless",
}
constants = config.get_model_constants(model_settings)
locals_d.update(constants)

Expand All @@ -175,6 +180,7 @@ def _location_sample(
chunk_size=chunk_size,
chunk_tag=chunk_tag,
trace_label=trace_label,
zone_layer=zone_layer,
)

return choices
Expand Down Expand Up @@ -235,13 +241,7 @@ def aggregate_size_terms(dest_size_terms, network_los, model_settings):
MAZ_size_terms = dest_size_terms.copy()

# add crosswalk DEST_TAZ column to MAZ_size_terms
maz_to_taz = (
network_los.maz_taz_df[["MAZ", "TAZ"]]
.set_index("MAZ")
.sort_values(by="TAZ")
.TAZ
)
MAZ_size_terms[DEST_TAZ] = MAZ_size_terms.index.map(maz_to_taz)
MAZ_size_terms[DEST_TAZ] = network_los.map_maz_to_taz(MAZ_size_terms.index)

MAZ_size_terms["avail_MAZ"] = np.where(
(MAZ_size_terms.size_term > 0)
Expand Down Expand Up @@ -368,6 +368,7 @@ def location_presample(
chunk_size,
chunk_tag,
trace_label,
zone_layer="taz",
)

# print(f"taz_sample\n{taz_sample}")
Expand Down Expand Up @@ -590,7 +591,13 @@ def run_location_simulate(
skim_dict = network_los.get_default_skim_dict()
skims = skim_dict.wrap("home_zone_id", alt_dest_col_name)

locals_d = {"skims": skims, "segment_size": segment_name}
locals_d = {
"skims": skims,
"segment_size": segment_name,
"orig_col_name": skims.orig_key, # added for sharrow flows
"dest_col_name": skims.dest_key, # added for sharrow flows
"timeframe": "timeless",
}
constants = config.get_model_constants(model_settings)
if constants is not None:
locals_d.update(constants)
Expand Down Expand Up @@ -895,6 +902,8 @@ def iterate_location_choice(

logger.debug("%s max_iterations: %s" % (trace_label, max_iterations))

choices_df = None # initialize to None, will be populated in first iteration

for iteration in range(1, max_iterations + 1):

persons_merged_df_ = persons_merged_df.copy()
Expand Down Expand Up @@ -934,9 +943,7 @@ def iterate_location_choice(
and iteration > 1
):
# if a process ends up with no sampled workers in it, hence an empty choice_df_, then choice_df wil be what it was previously
if len(choices_df_) == 0:
choices_df = choices_df
else:
if len(choices_df_) != 0:
choices_df = pd.concat([choices_df, choices_df_], axis=0)
choices_df_index = choices_df_.index.name
choices_df = choices_df.reset_index()
Expand Down
Loading