Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bug: Digest::Base cannot be directly inherited in Ruby #979

Closed
baurmatt opened this issue Aug 27, 2019 · 4 comments · Fixed by #980
Closed

Bug: Digest::Base cannot be directly inherited in Ruby #979

baurmatt opened this issue Aug 27, 2019 · 4 comments · Fixed by #980

Comments

@baurmatt
Copy link
Contributor

Describe the Bug

r10k sometimes throws an Digest::Base cannot be directly inherited in Ruby error. This leads to module not being deployed and therefore other errors (failing CI pipelines in our case).

This showed up after we enabled the pool_size feature. (Currently using pool_size: 10)

Expected Behavior

r10k doesn't fail and successfully deploys all modules.

Steps to Reproduce

Currently I can't reliably reproduce this. We had 6 (out of probably thousands) jobs failing with this error within ~24h.

Environment

  • Ruby: 2.5.3
  • r10k Version: 3.3.1
  • Platform: Debian 9.8 running in a docker container onto of a OpenStack VM.

Additional Context

Add any other context about the problem here.
@baurmatt
Copy link
Contributor Author

Running r10k with --trace surprisingly fast returned this stack trace:

ERROR	 -> Digest::Base cannot be directly inherited in Ruby
/usr/local/lib/ruby/2.5.0/digest.rb:35:in `new'
/usr/local/lib/ruby/2.5.0/digest.rb:35:in `file'
/usr/local/bundle/gems/r10k-3.3.1/lib/r10k/forge/module_release.rb:119:in `verify'
/usr/local/bundle/gems/r10k-3.3.1/lib/r10k/forge/module_release.rb:90:in `install'
/usr/local/bundle/gems/r10k-3.3.1/lib/r10k/module/forge.rb:149:in `install'
/usr/local/bundle/gems/r10k-3.3.1/lib/r10k/module/forge.rb:48:in `sync'
/usr/local/bundle/gems/r10k-3.3.1/lib/r10k/action/puppetfile/install.rb:36:in `visit_module'
/usr/local/bundle/gems/r10k-3.3.1/lib/r10k/action/visitor.rb:24:in `visit'
/usr/local/bundle/gems/r10k-3.3.1/lib/r10k/module/base.rb:86:in `accept'
/usr/local/bundle/gems/r10k-3.3.1/lib/r10k/puppetfile.rb:217:in `block in visitor_thread'

@baurmatt
Copy link
Contributor Author

Looks like aerospike/aerospike-client-ruby#45 had the same problem. Accessing a digest by Digest::SHA256 isn't thread-safe. Digest(:SHA256) is the way to go.

Going to open up a PR for this.

baurmatt added a commit to syseleven/r10k that referenced this issue Aug 27, 2019
@baurmatt
Make Digest subclass lookup thread-save
99fe4cd 
Fixes puppetlabs#979.
@baurmatt baurmatt mentioned this issue Aug 27, 2019
Merged
cmd-ntrf added a commit to cmd-ntrf/forge-ruby that referenced this issue Sep 12, 2023
@cmd-ntrf
Make Digest subclass lookup thread-safe in lru_cache
4e9cb06 
r10k content_synchronizer uses Forge v3 API when installing a module:
https://github.com/puppetlabs/r10k/blob/4.0.0/lib/r10k/module/forge.rb#L95
https://github.com/puppetlabs/r10k/blob/4.0.0/lib/r10k/module/forge.rb#L128
https://github.com/puppetlabs/r10k/blob/4.0.0/lib/r10k/module/forge.rb#L176

r10k uses a thread pool to install the modules, therefore the Forge v3
API calls have to be thread safe, and so does the LruCache class used to
cache API responses.

If LruCache is left not thread safe, the same error as reported in
r10k issue #979 happens.
puppetlabs/r10k#979
@cmd-ntrf cmd-ntrf mentioned this issue Sep 12, 2023
Merged
@tskirvin
Copy link

I'm seeing this consistently now, and it's causing r10k runs to fail. Fully up-to-date r10k on RHEL 8, puppetserver-7.17.1-1.el8.noarch.

@tskirvin
Copy link

Same deal when I try it on puppetserver-8.6.1-1. And I should note that it's not happening every run, but if I run it against a few dozen branches, I'm seeing errors about 5-6 times.

Note that I'm not seeing this error on my puppetserver-7.9.1-1.el8.noarch hosts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Make Digest subclass lookup thread-save syseleven/r10k
2 participants
@tskirvin @baurmatt