Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Non-mandatory tour frequency occasionally removes row from choosers table #841

Closed
JoeJimFlood opened this issue Apr 2, 2024 · 1 comment
Labels
Bug Something isn't working/bug f

Comments

@JoeJimFlood
Copy link
Contributor

We've had a couple of ABM3 runs fail during non-mandatory tour frequency due to a record being removed from the choosers table when doing some of the initial data processing. In both cases, it was resolved simply by starting over the full run from scratch. This is the error message that appears:

Traceback (most recent call last):
  File "\\jupiter\abm\dev\activitysim\activitysim\core\mp_tasks.py", line 930, in mp_run_simulation
    run_simulation(queue, step_info, resume_after, shared_data_buffer)
  File "\\jupiter\abm\dev\activitysim\activitysim\core\mp_tasks.py", line 882, in run_simulation
    raise e
  File "\\jupiter\abm\dev\activitysim\activitysim\core\mp_tasks.py", line 879, in run_simulation
    pipeline.run_model(model)
  File "\\jupiter\abm\dev\activitysim\activitysim\core\pipeline.py", line 529, in run_model
    orca.run([step_name])
  File "C:\ProgramData\Anaconda3\envs\asim_baydag\lib\site-packages\orca\orca.py", line 2177, in run
    step()
  File "C:\ProgramData\Anaconda3\envs\asim_baydag\lib\site-packages\orca\orca.py", line 973, in __call__
    return self._func(**kwargs)
  File "\\jupiter\abm\dev\activitysim\activitysim\abm\models\non_mandatory_tour_frequency.py", line 174, in non_mandatory_tour_frequency
    expressions.assign_columns(
  File "\\jupiter\abm\dev\activitysim\activitysim\core\expressions.py", line 138, in assign_columns
    results = compute_columns(df, model_settings, locals_dict, trace_label)
  File "\\jupiter\abm\dev\activitysim\activitysim\core\expressions.py", line 112, in compute_columns
    results, trace_results, trace_assigned_locals = assign.assign_variables(
  File "\\jupiter\abm\dev\activitysim\activitysim\core\assign.py", line 381, in assign_variables
    raise err
  File "\\jupiter\abm\dev\activitysim\activitysim\core\assign.py", line 355, in assign_variables
    expr_values = to_series(eval(expression, globals_dict, _locals_dict))
  File "<string>", line 1, in <module>
  File "\\jupiter\abm\dev\activitysim\activitysim\abm\models\util\overlap.py", line 344, in person_available_periods
    availability = pd.Series(availability, index=persons.index)
  File "C:\ProgramData\Anaconda3\envs\asim_baydag\lib\site-packages\pandas\core\series.py", line 461, in __init__
    com.require_length_match(data, index)
  File "C:\ProgramData\Anaconda3\envs\asim_baydag\lib\site-packages\pandas\core\common.py", line 571, in require_length_match
    raise ValueError(
ValueError: Length of values (67125) does not match length of index (67126)

The expression that is causing the error is this one. I noticed that the expression before is identical except for continuous=False in the call of person_available_periods(), leading me to believe that the issue is occurring in this function, as the only additional step that is added when the continuous variable is set to True is that function being called.

@JoeJimFlood JoeJimFlood added the Bug Something isn't working/bug f label Apr 2, 2024
dhensle added a commit to SANDAG/activitysim that referenced this issue Apr 19, 2024
@dhensle
Copy link
Contributor

dhensle commented Apr 19, 2024

The issue seems to be with the line c = np.bincount(starts[:, 0]). The np.bincount function counts the number of occurrences of each value in the input array. If the maximum value in the input array is less than the length of the array, the output of np.bincount will be shorter than the input array.

To fix this I added a minlength parameter to np.bincount to ensure that the output is of the desired length, as see in #854.

It is still a little unclear exactly why this is happening. The issue can often go away when just re-running. I believe @JoeJimFlood is the only one to hit this error so far and it is just one case in a full model run that get's lost. It seems like perhaps some person has an extremely unlikely daily pattern where the starts / stops happen to cause the (diffs == 1) to never be true.

Regardless of the root cause, the fix seems to prevent the crash.

dhensle added a commit to SANDAG/activitysim that referenced this issue Apr 19, 2024
jpn-- pushed a commit that referenced this issue Apr 25, 2024
@dhensle dhensle closed this as completed Apr 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working/bug f
Projects
None yet
Development

No branches or pull requests

2 participants