Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Go rewrite #8

Merged
merged 17 commits into from
Oct 14, 2017
Merged

Go rewrite #8

merged 17 commits into from
Oct 14, 2017

Conversation

marshyski
Copy link
Collaborator

@marshyski marshyski commented Oct 14, 2017

This is a rewrite from Python to Go. It is backwards compatible with the REST interfaces and configurations minus the rate limiting (buffering was added to depreciate that feature). All libraries are vendored in this project so it can be checked out and built without pulling from internet sources. There are a couple of optimizations in this implementation (listed below). You should get next to no-blocking of uploading RPMs to a repo with this implementation. Still have to tweak the buffering to do batches in high-stress periods. Performance benefits should be at least 3-10x improvement and minus the operational hassles of a Python project, see issue #5

Performance benefits:

  1. No need to write / read JSON file on disk, cacheing of contents from the sqlite DB, cache is invalided when new RPM is added and successful read from sqlite is completed.
  2. No process is triggered after every request is made that was using subprocess pop. This is now handled by the buffer to control new RPMs being added.
  3. No need for WSGI server / workers to scale python API. Go implementation excels at this this even without NGINX to handle other magic.
  4. Even with cache'ing the runtime requires less memory and CPU cycles.

@kood1
Copy link
Member

kood1 commented Oct 14, 2017

@marshyski - can you write a quick description of the change or link to an issue? thanks.

@marshyski
Copy link
Collaborator Author

@kood1 updated description of PR.

@kood1 kood1 merged commit 62a64f1 into master Oct 14, 2017
@kood1
Copy link
Member

kood1 commented Oct 14, 2017

Thanks @marshyski. It looks like CI build is failing. Can you take a look?

@kood1
Copy link
Member

kood1 commented Oct 14, 2017

@marshyski nvm, it passed

@marshyski marshyski deleted the go-rewrite branch October 14, 2017 16:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants