Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

remote_file checksum issue with client 12.6.0+ #4742

Closed
Stromweld opened this issue Mar 22, 2016 · 12 comments
Closed

remote_file checksum issue with client 12.6.0+ #4742

Stromweld opened this issue Mar 22, 2016 · 12 comments

Comments

@Stromweld
Copy link
Contributor

Description

using remote_file resource with checksum on chef-clients 12.6.0 and higher always gets the same checksum ID even if you change files and resource fails to download files.

Chef Version

12.6.0 through 12.8.1. 12.5.1 and below are working as expected.

Platform Version

CentOS 6.7 x86_64 linux.

Replication Case

bootstrap a server with 12.5.1 client and the collectd-ng cookbook in the runlist and everything works fine. Then bootstrap another node with 12.6.0 or higher with same runlist and cookbook fails on the remote_file download due to checksum error.

Client Output

client run
https://gist.github.com/Stromweld/8b78349757f5df1f9eee

debug run
https://gist.github.com/Stromweld/0d4fb07593f1fe051a74

Stacktrace

https://gist.github.com/Stromweld/3267621bd0f7f96fdbaa
@lamont-granquist
Copy link
Contributor

debug run is just an empty run_list

@lamont-granquist
Copy link
Contributor

the checksum on the resource looks accurate:

% shasum -a 256 collectd-5.4.1.tar.gz
853680936893df00bfc2be58f61ab9181fecb1cf45fc5cddcb7d25da98855f65 collectd-5.4.1.tar.gz

i'm not sure how the checksum on the new_resource.checksum is getting set. since 12.4.0 we've only saved the target checksum in the final_checksum on the resource and not ever mutated new_resource.checksum:

92824ed

@lamont-granquist
Copy link
Contributor

found where the checksum is getting set, so that is accurate:

https://github.com/hectcastro/chef-collectd/blob/b67c1d85a356774d3accfaed9ab9eee0d18254ff/attributes/default.rb#L4-L5

@lamont-granquist
Copy link
Contributor

#3991 did change between 12.5.1 and 12.6.0

@Stromweld
Copy link
Contributor Author

I fixed the debug output gist with a debug run with correct runlist.

Other thing I noticed is that even when I wrap this cookbook and change the file version and checksum to a newer version of collectd tar ball it throws the same error but with the same checksum so I think something is getting stored wrong or wrong file is getting checked that the computed sum doesn't change even if the file has.

172.21.26.171 Error executing action create_if_missing on resource 'remote_file[/var/chef/cache/collectd-5.4.1.tar.gz]'
172.21.26.171 ================================================================================
172.21.26.171
172.21.26.171 Chef::Exceptions::ChecksumMismatch
172.21.26.171 ----------------------------------
172.21.26.171 Checksum on resource (853680) does not match checksum on content (07d03e)

172.21.26.171 Error executing action create_if_missing on resource 'remote_file[/var/chef/cache/collectd-5.5.1.tar.gz]'
172.21.26.171 ================================================================================
172.21.26.171
172.21.26.171 Chef::Exceptions::ChecksumMismatch
172.21.26.171 ----------------------------------
172.21.26.171 Checksum on resource (4dba51) does not match checksum on content (07d03e)

@lamont-granquist
Copy link
Contributor

cool, i was trying to figure out how to ask you for information like that. it'd be nice to know what is in that tempfile but its difficult to check the tempfile since it is deleted automatically when the remote_file provider gets unwound by the stacktrace.

@lamont-granquist
Copy link
Contributor

Uh, looks like this isn't a bug in the checksum code:

[2016-03-23T14:21:52-04:00] DEBUG: ---- HTTP Response Body ----
[2016-03-23T14:21:52-04:00] DEBUG: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved <a href="http://collectd.org/temp/ContactMe.chef">here</a>.</p>
<hr>
<address>Apache/2.2.22 (Debian) Server at collectd.org Port 80</address>
</body></html>

[2016-03-23T14:21:52-04:00] DEBUG: ---- End HTTP Response Body -----

and that URL:

Hi,

you're using a Chef cookbook that downloads and builds collectd regularly. Due
to the size of the tarball, the popularity of the cookbook and traffic limits
on collectd.org this is causing problems for me.

I'm working with Miah Johnson, the original author of the code in question,
Noah Kantrowitz, the original author of the cookbook and others to resolve this
issue to everybody's satisfaction. If this is affecting you, please follow the
Github issue, where we track progress in this matter:

  https://github.com/miah/chef-collectd/issues/15

Also refer to the collectd mailing list to follow along the discussion and
voice your opinion. You can subscribe at:

  http://mailman.verplant.org/listinfo/collectd

Once this issue is resolved, please upgrade your cookbook. In the meantime we
kindly ask you to mirror the tarball locally. Thank you!

Sincerely,
—octo

Looks like we correctly blocked you from deploying a file that had been MITM'd (lol)

@lamont-granquist
Copy link
Contributor

and that repo has been deleted, and that redirect has likely been in place for a very long time.

i think what changed in 12.6.0 is that we fixed the User-Agent for chef requests which had been incorrectly telling the world that it was knife and went back to correctly telling the world it was chef-client...

@lamont-granquist
Copy link
Contributor

yeah #4074 was mitigated by 0103928#diff-19a9dc2400e2a14914853754a8083767

@Stromweld
Copy link
Contributor Author

ok, yeah looks like that is the issue after downloading the file and putting it on a local file server and changing download url it is working as expected. Sorry for the troubles.

@lamont-granquist
Copy link
Contributor

I cut collectd/collectd#1632 to see if we can't work through the issues with collectd.org folks again...

@Stromweld
Copy link
Contributor Author

cool thanks.

@chef chef locked and limited conversation to collaborators Nov 16, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lamont-granquist @Stromweld