-
Notifications
You must be signed in to change notification settings - Fork 100
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Joint Tour Chunking - Household Participants Need to Remain in Same Chunk #288
Comments
fixed this in develop by adding chunking by chunk_id, and updated tests to catch such errors in the future by forcing singleton full model run test to use minimal chunk size. This is provisional code as the multiple_zone branch has more extensive changes to chunking (adaptive chunking). |
@danielsclint to now test |
I see the new method in chunk to chunk by chunk id, but I don't see any changes to the joint tour participation code in develop. As this be merged with the main repo or in some other working version of develop? |
Oops! Sorry! It helps to push - which I just did. |
…pdate tests to catch such errors
* Provide more flexibility for defining mandatory schedule specifications. * provide more flexibility for time periods. * Improve consistency in slicing and fix code syntax issues. * Consistency and code syntax. * code syntax. * code syntax fixes. * syntax fix. * Add necessary change to test script settings. * Modify relevant section of the settings.yml for tests and examples. * restore 'hours' as deprecated for backwards compatibility. * Add default value for period_minutes for backwards compatibilitiy. * create develop branch so we can merge features here before merging (releasing) to master * provide more flexibility for time periods. * Improve consistency in slicing and fix code syntax issues. * Consistency and code syntax. * code syntax fixes. * Modify relevant section of the settings.yml for tests and examples. * restore 'hours' as deprecated for backwards compatibility. * Add default value for period_minutes for backwards compatibilitiy. * Fix overflows * Complete testing of the flexible time periods. * pep8 * test some floats too. * pep8 * start with csvs and write hdf5 outputs if desired (#290) * add h5_store option to write_tables * keep csv files as the default * read input tables from CSV * update docs, drop scripts no longer needed, include example files in package * formatting Co-authored-by: Blake <brosenthalpdx@gmail.com> * Update travis.yml * Pandas 1.0 fixes * Location and mode choice logsums (#298) * optional logsums for location_choice and tour destination models * mode choice logsums for trous and trips * trip DEST_CHOICE_SAMPLE_TABLE_NAME * update docs for writing logsums and also a couple small development page updates * fix small py2 py 3 compatibility issue * py2 py3 compatibility Co-authored-by: Blake <brosenthalpdx@gmail.com> Co-authored-by: Jeff Doyle <toliwaga@gmail.com> * Deprecate Py2 and add CLI * Tncs (#14) * add tncs * lognormal_for_df broadcasts duplicate indexes * lognormal_for_df broadcasts duplicate indexes code * correct lognormal calculations and add functions to calculation lognormal location and scale * clean-up tnc wait time expressions * set taxi, tnc wait time standard deviations to zero until we have better data * add scaled option to random.lognormal_for_df * add mobility-as-a-service modes to docs * updated tnc mode constants * rename maas to ridehail Co-authored-by: Jeff Doyle <toliwaga@gmail.com> Co-authored-by: Blake Rosenthal <blake.rosenthal@rsginc.com> * write trip matrices (#311) require newer pandas add ability to save trips table to pipeline and begin to finish example all time periods fix trip scheduling bug add support for odt skims as well Co-authored-by: Ben Stabler <benstabler@yahoo.com> Co-authored-by: Blake Rosenthal <blake.rosenthal@rsginc.com> * small fixes for semcog deployment (#319) * improved validation diagnostics in trip_purpose and various windows-related int32/int64 conversions * pycodestyle * deprecations in test code * fix bug in handling of no viable trips case in choose_trip_destination * add trace folder to example_mtc output folder Co-authored-by: Jeffrey Doyle <jeff.doyle@rsginc.com> Co-authored-by: Jeff Doyle <toliwaga@gmail.com> * major work on phase 5 (#325) * estimation through atwork_subtour_mode_choice * Tnc updates and notebooks (#18) * move other resources into folder since examples now part of package as well * add shapefile * clean up folder setup * estimation notebooks for larch (#19) * multiprocessing related logging and error checking * trip_destination handle all trips fail * skim caching with numpy memmap to speed skim loading * better chunking in vectorize_tour_scheduling * Cli (#22) * use activitysim_resources Fixed auto sufficiency conditions in tour_mode_choice.csv. (Issue #324) * increment to version 0.9.5 * correct write trip matrices sampling expansion and add vehicle occupancy to the expression file * updates to documentation for phase 5 work except multiple zone systems * additional updates for estimation integration (#328) * estimation integration updates #327 * drop module no longer needed * update license, fix univ coeff bug, update estimation example (#339) * improve example setup for specifying input tables (#27) * improve LICENSE (#30) * correct univ coeff template lookup (#28) * add license to package (#341) * move notebooks to examples folder so they are installed and update docs as a result (#343) * correct doc links * Address Minor Issues Associated with ARC Update to Develop Branch (#342) * publish v0.9.5 (#340) * update docs, drop scripts no longer needed, include example files in package * Pandas 1.0 fixes * Location and mode choice logsums (#298) * Tncs (#14) * write trip matrices (#311) * small fixes for semcog deployment (#319) * improved validation diagnostics in trip_purpose and various windows-related int32/int64 conversions * fix bug in handling of no viable trips case in choose_trip_destination * major work on phase 5 (#325) * estimation through atwork_subtour_mode_choice * Tnc updates and notebooks (#18) * move other resources into folder since examples now part of package as well * add example zone shapefile * estimation notebooks for larch (#19) * multiprocessing related logging and error checking * trip_destination handle all trips fail * skim caching with numpy memmap to speed skim loading * better chunking in vectorize_tour_scheduling * Cli (#22) * use activitysim_resources Fixed auto sufficiency conditions in tour_mode_choice.csv. (Issue #324) * correct write trip matrices sampling expansion and add vehicle occupancy to the expression file * additional updates for estimation integration (#328) * improve LICENSE (#30) * correct univ coeff template lookup (#28) Co-authored-by: Clint Daniels <clint.daniels@wsp.com> Co-authored-by: Blake <brosenthalpdx@gmail.com> Co-authored-by: Blake Rosenthal <blake.rosenthal@rsginc.com> Co-authored-by: Jeff Doyle <toliwaga@gmail.com> Co-authored-by: Jeffrey Doyle <jeff.doyle@rsginc.com> Co-authored-by: Jeffrey Newman <jeff@newman.me> * Remove duplicate coefficient. * move coefficient processing up * Update joint_tour_destination.py * Fix reference to joint tour destination.yaml * Error check for COEFFICIENT files * Add person_id to joint choosers. * move notebooks to examples folder so they are installed and update docs as a result (#343) * publish (#344) * move notebooks to examples folder so they are installed and update docs as a result (#343) * correct doc links * Remove duplicate coefficient. * move coefficient processing up * Update joint_tour_destination.py * Fix reference to joint tour destination.yaml * Error check for COEFFICIENT files * Add person_id to joint choosers. * Remove duplicate coefficient. * move coefficient processing up * Update joint_tour_destination.py * Fix reference to joint tour destination.yaml * Error check for COEFFICIENT files * Add person_id to joint choosers. Co-authored-by: Ben Stabler <bstabler@users.noreply.github.com> Co-authored-by: Blake <brosenthalpdx@gmail.com> Co-authored-by: Blake Rosenthal <blake.rosenthal@rsginc.com> Co-authored-by: Jeff Doyle <toliwaga@gmail.com> Co-authored-by: Jeffrey Doyle <jeff.doyle@rsginc.com> Co-authored-by: Jeffrey Newman <jeff@newman.me> Co-authored-by: bstabler <benstabler@yahoo.com> * fix #288 chunk joint_touyr_participation by chunk_id and update tests to catch such errors * gitignore trivia * Support for Multiple Zone Systems and Transit Virtual Path Building (#362) * TransitVirtualPathBuilder first cut - get_tvpb_logsum computes logsums on demand * tour_mode_choice logsums for 3-zone working for school and workplace location * z3 running all models, but with hacked trip_mode_choice (no coefficient templates) * z3 running all models * passing tests * pcodestyle passes * refactored test teardown * tvpb tracing * tvpb tracing roughout * build multizone test data before running test * read_settings_file support include_settings * read_settings_file support include_settings * move constants to configs yaml and expressions from abm.util to core * tvpb estimate_chunk_overhead * adaptive chunking * fixed bug in trip scheduling when no trips in leg after iteration * 3 zone tap_tap uniquify * tvpb TableCache * tvpb TableCache - all fiiels * 3 zone with feather cache * tidy up notes around creating the marin example * memmap skims * memmap skims * correct drive transit cost expression for miles/feet. Still need to add some additional missing expressions now that everything appears to be working correctly. * skim docstrings * disable 3 zone tap_tap caching for traced taps * initialize_los * 3 zone mp plumbing untested * THREE_ZONE tvpb multiprocessing tests and fixes * THREE_ZONE tvpb multiprocessing tests and fixes * reanme core tvpb and cache modules * reanme core tvpb and cache modules * util.iprod alias for np.prod with dtype int64 * THREE_ZONE cache initialization bug bixes * multizone bug fixes * bugs * multiprocessing mjultizone tweaks * tvpb cacha\e as array * complain if apporting with more process than slice rows * complain if apporting with more process than slice rows * chunk initialize_tvpb * shorter trace file names because windows * refactor pathbuilder sans DYNAMIC * fix tour_scheduling_calc_row_size * fix multiprocess resume across mp step bug * pycodestyle * minor chunk cleanup * allow MemMapSkimFactory multiprocessing * consolidate mtc and multizone configs * commit updated marin example config files * some more marin example config updates * add validation summaries script * some additional marin example updates * update example skim time period settings to align with tm1 (#360) * update example settings to align with tm1 * correct home_is_rural coding since tm1 was incorrect * corrected skim time periods go in network_los.yaml now * travis depends on these scripts for testing * improve logging and increase travis timeout * #359 (#364) * updates for CDAP person type mapping * allow duplicate time labels * fix a copy small items for #359 * test updates * pycodestyle * 3 zone test fix Co-authored-by: Clint Daniels <clint.daniels@wsp.com> * longer travis timeout and avoid file permissions between tests related to caching * test-travis * ls fix * add timeout * fix path * put in both locations for now * reduce household sample sizes for tests to avoid travis timeout * not sure what's taking so long with the second and third tests. It runs in 40 minutes with py 3.7 on my computer as well * linux file paths * cli test depends on py 3.7+ * add coveralls back * test new architecture * try build matrix * setup environments * correct matrix expansion * correct duplicate key * correct key * correct env usage * no need to specify architecture * update inject settings for tests * updated examples to use activitysim resources repo * rename folder * update survey test files so all example_manifest.yaml examples now running * updated notebooks, default settings for examples, and updated rst files * fix a couple typos * add comments to travis script * ARC Updates (#365) Add parking_location_choice, trip_departure_choice, and trip_scheduling_choice models. * add example_mtc_arc_extensions so arc extensions can be run and tested * small doc update * fix arc extensions example Co-authored-by: Clint Daniels <clint.daniels@wsp.com> Co-authored-by: Blake <brosenthalpdx@gmail.com> Co-authored-by: Blake Rosenthal <blake.rosenthal@rsginc.com> Co-authored-by: Jeff Doyle <toliwaga@gmail.com> Co-authored-by: Jeffrey Doyle <jeff.doyle@rsginc.com> Co-authored-by: Jeffrey Newman <jeff@newman.me>
In the joint tour participation model, its possible for the chunker to split potential joint tour participants from the same household across two different chunks. This will create an endless loop and eventual assertion error in the joint tour participation model.
activitysim/activitysim/core/chunk.py
Lines 217 to 229 in 7b57c94
If a household is split, so only one person is chosen in the chunk, the following code will never evaluate to true, because
x.participants > 1
will always evaluate toFalse
. Or...x.participants > x.adults
will always evaluate toFalse
.activitysim/activitysim/abm/models/joint_tour_participation.py
Lines 86 to 87 in 7b57c94
Eventually, the mode will crash in the following set of code after the maximum number of iterations for finding participants occurs.
activitysim/activitysim/abm/models/joint_tour_participation.py
Lines 158 to 165 in 7b57c94
The text was updated successfully, but these errors were encountered: