Slow performance with prpy 0.4.0 #26
Comments
|
We recently fixed a but (see personalrobotics/prpy#79) that was causing environment cloning to leak a significant amount of memory. This could be the culprit if you're the demo in a loop in one process. It doesn't look like that commit made it into the 0.4.0 release. Can you try the latest If that doesn't fix the problem, then you'll need to do some lower-level profiling to see what is actually the slow part. Is it planning or retiming/smoothing? |
|
BTW, here is some documentation for how to make a proper table on Github using Markdown. The original timing information was very difficult to read, so I edited your post accordingly. 😄 |
|
Btw, here are the results from the collision and kinematics benchmarks of or_benchmarks: |
|
Those results don't mean much because there is (likely) nothing in the environment. It would be more informative to run the self-collision benchmark. I believe the |
|
Hmm the run_all script relies on some datasets that appear to have been generated for Herb: Thanks for the reference on git-tables :) |
|
Also, checking out the latest prpy results in the following error: |
|
In that case, you can just run the standalone @jeking04 Could you take a look at making |
|
@Stefanos19 You should file an issue on PrPy. Even if the error is in AdaPy, there is no excuse for PrPy throwing a |
|
Here is the output of run_collision_benchmark with the --self option: |
|
@mkoval ok, done. |
|
That is 2950.79 self-collision checks per second. This is quite a bit faster than HERB using ODE. It's hard to draw any concrete conclusion from that, since this is partially expected:
We may want to try the latest version of PrPy (once personalrobotics/prpy#89 is fixed) and do some more profiling of the demo script before putting too much time into simplifying the model. |
This may be true, but sometimes the opposite can be true for a different reason -- the checker can return False on the first collision it finds, whereas if the configuration is valid, it must finish checking all links before it returns True. I have no idea what the tradeoff is for the Ada model, though. |
|
@cdellin Yes, that is definitely true. The worst case is definitely a near-miss, but it's not clear how "in collision" versus "not in collision" compare. |
|
I ran the bite-serving code with the latest version of prpy. It is slightly faster than 0.4.0 (a bite cycle takes on average 37 secs with the latest master vs 41 secs for the 0.4.0), but it is still much slower than the prpy 0.3.1 (26 secs). |
|
There shouldn't be any changes in PrPy between 0.3.1 and Looking at your table of times, it looks like the big increase in time was split between |
|
@Stefanos19 Re: the |
|
I uploaded the file that I used for testing. In all tests I used only the cbirrt planner for everything, as well as the openravepy smoother, for consistency. To ensure that only the cbirrt planner is called, I changed the Sequence in adarobot.py, as follows: actual_planner = Sequence(
self.birrt_planner,
MethodMask(
FirstSupported(
self.cbirrt_planner,
),
methods=['PlanToIK', 'PlanToTSR', 'PlanToEndEffectorPose', 'PlanToEndEffectorOffset']
)
)
self.planner = FirstSupported(
actual_planner,
# Special purpose meta-planner.
)For the prpy 0.3.1 test, I also changed the sequence in adarobot.py so that The times are the times from the first PlanToEndEffectorOffset, when the robot starts to go grab a bite, until it reaches the same point again, after it serves the bite to the person. |
|
@mkoval On the PlanToTSR error, I found the bug and deleted the comment, but it was too late :) Thanks!! |
|
It's hard to make much of those numbers. Could you:
with prpy.util.Timer('PlaningToXXX'):
path = robot.PlanToXXX(..., execute=False)
with prpy.util.Timer('MovingToXXX'):
robot.ExecutePath(path)Also, I would expect CBiRRT to be very slow with |
|
@Stefanos19 Was the |
|
@mkoval It was in my code, I used the "old" sequence formatting which caused the error. |
|
@Stefanos19 What is the old sequence formatting? I don't think |
|
In general, we need to figure out which specific function call(s) in PrPy got slower in |
|
Sorry, I did not phrase this well. I meant that in 0.3.1, I did: |
|
That should definitely not cause an error. This most likely means that CBiRRT is always failing because of a syntax error and another planner is picking up the slack. Can you file an issue on PrPy? BTW: There is no reason to use |
|
Ok, I will replicate the error to make sure that this was it. It is possible that it was something else. |
|
Yes, that is correct. But |
|
Now that I think about it, it is possible that the TSR error was caused because I did not set up the actual_planner = Sequence(
self.birrt_planner,
MethodMask(
FirstSupported(
self.cbirrt_planner,
),
methods=['PlanToIK', 'PlanToTSR', 'PlanToEndEffectorPose', 'PlanToEndEffectorOffset']
)
)
self.planner = FirstSupported(
actual_planner,
# Special purpose meta-planner.
) |
|
In any case, it is a bug if CBiRRT is crashing with that error. You should report it on PrPy. I also don't see why that sequence would fix anything - CBiRRT is the only planner in the list that implements BTW: That list is equivalent to: self.planner = FirstSupported(
self.birrt_planner,
self.cbirrt_planner
) |
|
Got it. I will replicate the error and file the bug, and redo the comparison using self.planner = self.cbirrt_planner in both and with the time-tests that you have above. |
|
Make sure you get timing information for the individual planning calls. We really do need to figure out what part of PrPy got slower. |
|
Right. |
|
I think I fixed the CBiRRT bug here: personalrobotics/prpy@e7b48b8 |
|
That's interesting: Replacing self.manip.PlanToConfiguration(servingConfiguration)with path = self.robot.PlanToConfiguration(servingConfiguration, execute=False)
self.robot.ExecutePath(path)solved the timing issue. Aren't the two identical? |
|
They should be identical. Passing I don't know why there is a difference if you call it manually. I am a bit surprised that this works at all. There is a bunch of HERB-specific code in the |
|
BTW: Your first snippet calls the planner on |
|
What?! How does manip even have any of those planning methods? Aren't they all inherited from robot base? |
|
They're magically added to The only difference is that calling a planning method on a manipulator first sets itself as active. |
|
Aha. So what is with this slowdown then? |
|
Is this fixed? What was the problem? |
|
I am not sure what the problem was. I no longer see the slowdown between calling path = PlanToConfiguration(config, execute=False), ExecutePath(path), and PlanToConfiguration(config). It might have been the ancient or_ompl version I used, or the smoothing/shortcutting parameters. |
The two videos below run identical code, apart from the the first using prpy 0.3.1 and the second 0.4.0 For the first, the duration of grasp_bite - serve - look - ready-to-grasp is 25sec, for the second 49 secs https://www.youtube.com/watch?v=YkwINudZnOw https://www.youtube.com/watch?v=LbgmtMS4kpo
I took time stamps before and after each PlanToConfiguration and PlanToEndEffectorOffset call:
It appears though that the actual delays in the video are much larger than the ones in the time stamps
@siddhss5 @mkoval @jeking04
The text was updated successfully, but these errors were encountered: