-
Notifications
You must be signed in to change notification settings - Fork 100
Project Meeting 2024.02.15
Michelle Bina edited this page Feb 19, 2024
·
5 revisions
- Update on Phase 9a Optimization task
- Recent pull requests
- Admin
- AMPO Conference
- TO DO: RSG to fold the MWCOG improvements into the MTC extended model and test that with sharrow and see if that address the run time problems.
- TO DO: Jeff to work on explicit chunking to see if that addresses the memory issues with the no sharrow version of the code.
- TO DO: Sijia to look into getting the non-sharrow version of the code to run for the MTC extended model.
- TO DO: CS to send out an initial cut at a proposal for AMPO conference and send out to others.
- All bench contractors have tried to run the MTC extended model, full sample.
- When running without sharrow, everyone crashed with a memory error in vehicle type choice (but at this point, they don’t know how much would be required). Sijia was able to get 100% sample of the MTC extended model to run on a 512GB machine before this version of the code. Issue is probably the last PR that was merged that may have introduced a bug.
- Jeff was able to run through with sharrow in 8-9 hours, but it crashed near the end in trip destination. The crash was caused by a bug that was triggered with some failed trips. This bug was never found before because samples weren’t big enough to create failed trips. Jeff opened a pull request to fix that bug. All bench contractors should be able to run with sharrow on soon, after Jeff's bug fix.
- Sijia showed memory profiling charts on the steps that they were able to run. Jeff and Sijia’s charts are different - this may be attributed to Jeff running on a Mac versus Sijia running on windows. There are some differences on how Mac uses/reports memory, so it might not be an apples-to-apples comparison between Jeff and Sijia's memory profiles. A true comparison may not be possible.
- Is it possible to do more work on non-sharrow version? Is it worth investing time, if we can’t get it to run without sharrow? If the vast majority of models would be running with sharrow, would be wasting money to optimize without sharrow? Is it possible?
- It is possible – but is it worth it? Maybe try to spend a modest amount of effort (1-3 days) to do some profiling but not keep it wide open that it needs to be fixed.
- Can we just go ahead with improvements to the sharrow version and not wait until the non-sharrow version is at the same place?
- Jeff to lead the sharrow optimization. RSG can provide some things that they’ve been working on - coordinating with Jeff and creating PRs based on what they’ve done so far. (RSG has worked on removing strings, and Jeff will work to optimize how sharrow handles strings.)
- Sijia to dig into the non-sharrow version.
- Specifically looking at vehicle type choice and sharrow performance: it is the same before and after the data optimization for vehicle type choice and RSG recognized that it was recompiling each time it was run for the MWCOG implementation, which was causing very long run times.
- One of the outcomes of vehicle type choice is the type, age, and fuel type of the vehicle, concatenated as a string. There are a lot of different possible combinations – converting strings into categoricals can get a lot, but not all the possible combinations. New combinations can trigger sharrow to recompile, because it can become a new categorical. Need to think about how to solve that problem – either write out all the possible outcomes or something else? Sijia did write out all the possible combinations for other models to define the categoricals, not just those that show up in the sample, but didn’t do that for vehicle type choice. Another solution with vehicle type choice would be to do explicit chunking. PSRC is doing that now with aggregate accessibilities, which doesn’t help with run time but it works, but not optimally. Not the best long-term solution but might be a good short-term one.
- TO DO: RSG to fold the MWCOG improvements into the MTC model and test that with sharrow and see if that address the run time problems.
- TO DO: Jeff to work on the chunking to see if that addresses the memory issues with the no sharrow version.
- Do we want to do these things before merging in the BayDAG PRs? Yes.
- Structure of Repository
- Jeff opened an issue that describes the current structure.
- This may not be the best way to organize the system.
- The main branch should be the develop branch, where you can tag releases and still roll back to whatever release version.
- With this proposed change - when someone wants to make a contribution, they can make it where we are working, not where releases were made.
- At some point, the main branch will break. If someone checks out that branch at the moment when it’s broken, it may cause problems. Need to be clear to users that they need use tagged release versions. Needs to be clearly stated in README and documentation.
- How does this work in python repos? Theoretically, if there’s rigorous testing, changes won’t be merged unless it’s confirmed to be good. (That said, bugs can get in).
- With no objections – Jeff will make this change and add the warnings that were discussed.
- Change to ruff for formatting and code quality checks?
- There wasn't time to discuss. Folks can click on the link to read more, and we'll discuss later.
- For proposal, borrow/adapt from our last submittal and update for today.
- TO DO: CS to send out an initial cut at a proposal and send out to others.
- AMPO would like at least one or two agencies to present, not just consultants.
- MWCOG said they will participate.