added abstraction for bulk operations through redis pipeline #82
Conversation
| def set_bulk(self, payload): | ||
| """Write bulk entries to config db. | ||
| """ | ||
| client = self.redis_clients[self.db_name] |
There was a problem hiding this comment.
You need to use the below API which is Multi-DB compliant.
self.get_redis_client(self.db_name). Modify other occurrences as well.
|
@qiluo-msft can you please review this change. These bulk API's improve performance when performing large number of table operations. |
| @@ -193,6 +193,42 @@ def set_entry(self, table, key, data): | |||
| k = k + '@' | |||
| client.hdel(_hash, self.serialize_key(k)) | |||
|
|
|||
| def set_bulk(self, payload): | |||
There was a problem hiding this comment.
set_bulk [](start = 8, length = 8)
What are the use cases for the new functions? Application can achieve pipeline by calling get_redis_client().pipeline(). And that solution will be more flexible since you can mix functions into the same pipeline.
There was a problem hiding this comment.
Qi, Please refer to the below comment from. It was suggested not to use the pipeline in the application.
sonic-net/sonic-utilities#891 (comment)
The use case is also mentioned in the same PR. It is to create large number of VLANs. This also helps when querying and displaying the contents of a filled-up mac address table.
There was a problem hiding this comment.
Thanks @rajendra-dendukuri sharing the background! As we planed to move away from sonic-py-swsssdk and converge everything to sonic-swss-common, I believe new feature here will be short-lived. We have implemented some pipeline support in sonic-swss-common *Tables classes. Could you check if that fulfill your requirement?
Sorry for the inconvenience to extend code here.
In reply to: 445300357 [](ancestors = 445300357)
|
For new commands to add/del range of vlans and range of vlan members, added new interfaces for bulk redis pipeline update