Add program cache for executor.py #8744
Conversation
… add-program-cache-for-executor
python/paddle/fluid/executor.py
Outdated
| return_numpy=True, | ||
| use_program_cache=False): | ||
| """ | ||
| :param program: the program that need to run |
There was a problem hiding this comment.
This is core API. I would suggest more comments. For example, does the program run from begin to end, or does it just run the nodes in the fetch dependency.
There was a problem hiding this comment.
Even better, add a small python example of using Executor.
python/paddle/fluid/executor.py
Outdated
| :param program: the program that need to run | ||
| :param feed: feed variable list | ||
| :param fetch_list: fetch variable list | ||
| :param feed_var_name: feed_var_name default to 'feed' |
There was a problem hiding this comment.
Maybe a bit more comment here? I'm confused about the difference between "fetch_list" and "fetch_var_name". And the meaning of default 'feed'/'fetch'.
| @@ -225,7 +226,19 @@ def run(self, | |||
| feed_var_name='feed', | |||
| fetch_var_name='fetch', | |||
| scope=None, | |||
| return_numpy=True): | |||
| return_numpy=True, | |||
| use_program_cache=False): | |||
There was a problem hiding this comment.
I would remove this 'option' and use 'True' by default. Then as a follow up, auto-detect the program change (preferred) and auto invalidate cache.
There was a problem hiding this comment.
currently, the cache cannot check if the program has changed, so user need to set this flag if they are sure the program has not changed, so we have to set this to false by default.
python/paddle/fluid/executor.py
Outdated
| :param return_numpy: convert the fetched tensor to numpy | ||
| :param use_program_cache: set use_program_cache to true if program not changed compare to the last step. | ||
| :return: | ||
| """ |
python/paddle/fluid/executor.py
Outdated
|
|
||
| for op in global_block.ops: | ||
| if use_program_cache: | ||
| self.program_caches[program_cache_key] = program_cache |
There was a problem hiding this comment.
Maybe limit the number cached here? Save memory if the program change often.
There was a problem hiding this comment.
Will drop cache of program_cache_key when use_program_cache is False now.
|
BTW, in fact, our |
| @@ -177,6 +193,7 @@ def __init__(self, places): | |||
| # TODO(dzhwinter) : only use the first place | |||
| self.executor = core.Executor(act_places[0]) | |||
| self.places = places | |||
| self.program_caches = dict() | |||
| name=feed_var_name, | ||
| type=core.VarDesc.VarType.FEED_MINIBATCH, | ||
| persistable=True) | ||
| self.program_caches.pop(program_cache_key, None) |
| program_cache = program.clone() | ||
|
|
||
| if use_program_cache: | ||
| self.program_caches[program_cache_key] = program_cache |
There was a problem hiding this comment.
Leave a TODO here to avoid caching too many programs?
related issue: #8729