Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kvstore str interface (not related to sparse) #157

Closed
eric-haibin-lin opened this issue Aug 10, 2017 · 0 comments
Closed

kvstore str interface (not related to sparse) #157

eric-haibin-lin opened this issue Aug 10, 2017 · 0 comments

Comments

@eric-haibin-lin
Copy link
Owner

updater still uses the int interface, which is not consistent with the str kv.push/pull interface.
To have consistent view of keys, we should create a kvstore scheduler role (assume it never dies). The scheduler hold the str<->int mapping of all keys. Or we can have rank0 server responsible for this. It also needs to broadcast new key to all other servers. Each server will have to store the reverse mapping of int -> str.
eric-haibin-lin pushed a commit that referenced this issue Dec 9, 2019
@ymjiang
server: initial commit (#151)
c1b20f8 
* server: first commit

* server: quick fix

* server: fix building

* server: fix launcher

* server: add new server logic

* common: fix compile warning

* server: attempt to fix rdma compile

* server: add init_global_env()

* server: improve the log

* server: improve engine load balance

* server: use omp to speed up memcpy (#152)

* server: remove recvmap_mu_

* server: avoid global mutex

* server: remove recv->merged copy

* server: attempt to fix correctness

* server: PriorityQueue for server (#153)

* server: add priority queue to server

* server: quick fix for pq

* server: fix compile

* server: improve with heap

* server: fix disable schedule

* server: fix heap sorting correctness

* server: another fix of heap correctness

* 3rdparty: update pslite

* server: improve priority compare

* common: disable numa control when not summing locally (#157)

* docker: update dockerfile for new server

* docker: clean up dockerfiles

* docker: refactor and test all (#169)

* preliminary refactor for tf

* can compile for tf

* refactor mxnet and torch

* test all

* nit

* docs: clean examples and update tutorial

* docker: quick fix

* docs: fix tutorial

* docs: update performance

* docs: update readme

* docs: improve legend in readme
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@eric-haibin-lin