Documented how content trust affects VCHs #1850

stuclem · 2018-06-26T10:19:55Z

Related to #1659 in that I think that we should recommend against adding the same VCH to more than one project.

@zjs @hickeng can you please review? Thanks!

stuclem · 2018-06-26T14:19:50Z

Per comment from @pdaigle via Slack, we should recommend against adding the same VCH to multiple projects. 22f6ce7 fixes #1659.

zjs

These changes look fine, however I've never used content trust and can't find a functional specification or design document describing how it's intended to work, so I'm probably not the best reviewer for these changes.

zjs · 2018-06-26T15:45:04Z

docs/user_doc/vic_cloud_admin/add_vch_fullTLS_in_portal.md

@@ -2,7 +2,7 @@

 If a Docker host or a vSphere Integrated Containers virtual container host (VCH) implements verification of both server and client certificates, you must provide the details of the client certificate when you add the host to a project in the management portal. Connections to the host use HTTPS.

-**IMPORTANT**: If you have deployed multiple instances of the vSphere Integrated Containers appliance, you can only register a virtual container host (VCH) with one instance of the management portal at a time.
+**IMPORTANT**: You can only register a VCH with one project at a time. Similarly, you cannot add the same VCH to projects in multiple instances of vSphere Integrated Containers.


Is this something Admiral enforces, or is this the user's responsibility to ensure? If the latter, is there an easy way for a user to see if the VCH is already registered with a project?

@zjs in short, no, it's not enforced. There is no way of knowing whether a VCH is already in a project.

@pdaigle told me yesterday that he has requested that this be enforced in a future release.

Also, @zjs, I based my testing on the test spec for CT: https://github.com/vmware/vic-product/blob/master/tests/manual-test-cases/Group2-OVA-Features/2-06-Trusted-Content.md

@stuclem It may be related to the doc guidelines, but I would have said should not or must not in the style of RFCs. To me, cannot implies that you are prevented from doing so rather than I take on the burden of policing the requirement.

I don't know if a non-VCH can be added to projects these days, and whether they can be mix-and-matched if so.

@hickeng agree about "should". Fixed in b3c0052.

And yes, I do believe that you can still mix-n-match VCHs and Docker hosts in a project, although I have not tested this.

stuclem · 2018-06-27T08:23:08Z

Thanks for the approval, @zjs. Do we need @hickeng to take a look too before I merge?

hickeng · 2018-06-29T00:16:23Z

docs/user_doc/vic_cloud_admin/add_vch_fullTLS_in_portal.md

@@ -2,7 +2,7 @@

 If a Docker host or a vSphere Integrated Containers virtual container host (VCH) implements verification of both server and client certificates, you must provide the details of the client certificate when you add the host to a project in the management portal. Connections to the host use HTTPS.

-**IMPORTANT**: If you have deployed multiple instances of the vSphere Integrated Containers appliance, you can only register a virtual container host (VCH) with one instance of the management portal at a time.
+**IMPORTANT**: You can only register a VCH with one project at a time. Similarly, you cannot add the same VCH to projects in multiple instances of vSphere Integrated Containers.


@stuclem It may be related to the doc guidelines, but I would have said should not or must not in the style of RFCs. To me, cannot implies that you are prevented from doing so rather than I take on the burden of policing the requirement.

I don't know if a non-VCH can be added to projects these days, and whether they can be mix-and-matched if so.

hickeng · 2018-06-29T00:21:18Z

docs/user_doc/vic_cloud_admin/content_trust.md

+
+vSphere Integrated Containers Registry provides a Docker Notary server that allows you to implement content trust by signing and verifying the images in the registry. 
+
+Content trust is enabled at the project level. If content trust is enabled on a project, users can only push and pull images to and from that project if they are signed and verified by the Notary server. 


It may be worth expanding on this further for people who are familiar with Notary and say:

The registry will refuse to serve images to a client from a project with content trust enabled if they are not signed and verified.

and a statement about how to mix signed and unsigned images - I'm not actually sure how best to do that; the implication is that a separate project is required, but given a project is also an RBAC boundary that may mean creating two projects for a given set of users. Can ask admiral or harbor if the public/private/global mechanisms provide a path for this.

@hickeng agree about "should" and added your extra sentence in b3c0052.

Opened #1863 to find out about mixing signed and unsigned images.

hickeng · 2018-06-29T00:25:18Z

docs/user_doc/vic_cloud_admin/content_trust.md

+
+Content trust is enabled at the project level. If content trust is enabled on a project, users can only push and pull images to and from that project if they are signed and verified by the Notary server. 
+
+Enabling content trust on a project automatically modifies the registry whitelist settings of any virtual container hosts (VCHs) that are registered with the project. Consequently, when content trust is enabled, the VCHs in the project can only pull signed and verified images from the registry instance that is running in the vSphere Integrated Containers appliance. 


Addendum:

Updating the whitelist settings of the VCH via content trust requires that the existing whitelist settings of the VCH permit pulling from the registry. The VCH will reject the content trust update if it would result in a more permissive configuration than currently configured by the administrator.

Added in b3c0052.

hickeng · 2018-06-29T00:30:14Z

docs/user_doc/vic_cloud_admin/content_trust.md

+
+Enabling content trust on a project automatically modifies the registry whitelist settings of any virtual container hosts (VCHs) that are registered with the project. Consequently, when content trust is enabled, the VCHs in the project can only pull signed and verified images from the registry instance that is running in the vSphere Integrated Containers appliance. 
+
+If you enable content trust on a project, the developers who work on that project must configure their Docker clients so that they can connect to the Notary server running in vSphere Integrated Containers Registry. 


I'm not sure this is actually required - I think developers may configure their clients for notary but as the registry will not serve unsigned images it's done for the developer to assert their own trust rather than the infrastructure.

I don't know if it's worth noting - might just get confusing.

Deleted this para in b3c0052.

hickeng · 2018-06-29T00:40:58Z

docs/user_doc/vic_cloud_admin/vchs_and_mgmt_portal.md


-**NOTE**: vSphere Integrated Containers Management Portal allows you to provision containers from the registries that are included in the lists of   global registries that the Management Portal Administrator configures, or project registries that the DevOps administrator configures. However, if the vSphere administrator deployed a VCH with whitelist mode enabled, and if the whitelist on the VCH is more restrictive than the global and project registry lists, you can only provision containers from the registries that the VCH permits in its whitelist, even if the VCH is included in a project that permits other registries. For more information, see [VCH Whitelists and Registry Lists in vSphere Integrated Containers Management Portal](../vic_vsphere_admin/vch_registry.md#vch-whitelist-mp) in *vSphere Integrated Containers for vSphere Administrators*.
+**IMPORTANT**: vSphere Integrated Containers Management Portal allows you to provision containers from the registries that are included in the lists of global registries that the Management Portal Administrator configures, or from project registry lists that the DevOps administrator configures. However, if the vSphere administrator deployed a VCH with whitelist mode enabled, and if the whitelist on the VCH is more restrictive than the global and project registry lists, you can only provision containers from the registries that the VCH permits in its whitelist, even if the VCH is included in a project that permits other registries. Also, if you add a VCH to a project on which content trust is enabled, vSphere Integrated Containers updates the whitelist configuration of the VCH. As a consequence, you should only add a given VCH to one project at a time. Adding the same VCH to multiple projects can lead to conflicts if the registry lists and content trust setttings are different in the different projects.


I don't know what is meant by a conflict when adding a VCH to multiple projects.

When a VCH is added to a project it tightly binds to both that specific VIC appliance and project. I believe, but have not confirmed, that if you were to add the VCH to a second project the VCH would never know and there would be zero impact on it. The VCH whitelist would be configured for that first project only.
If you then removed the VCH from the first project the VCH would likely discover itself in the second project and rebind to that.

@hickeng, yes, "conflict" is not the right word. On re-reading, I think that this sentence is redundant anyway, given what is said in the preceding text. Removed the sentence in b3c0052.

hickeng · 2018-06-29T00:57:31Z

docs/user_doc/vic_vsphere_admin/vch_registry.md

+VCHs include a base set of well-known certificates from public CAs. If a registry requires a certificate to authenticate access, and if that registry does not use one of the CAs that the VCH holds, you must provide the CA certificate for that registry to the VCH. If the VCH is running in whitelist mode, you must also add that registry to the whitelist.
+
+- If you provide a registry certificate but you do not also specify that registry in the whitelist, the VCH does not allow access to that registry. 
+- If you specify a registry in the whitelist, but you do not provide a certificate and the registry's CA is not in the set of well-known certificates in the VCH, the VCH does not allow access to that registry.


If you specify a registry as insecure (see below) then it is added to the whitelist.

(assuming I recall the behaviour correctly).

Added in b3c0052.

hickeng · 2018-06-29T01:00:53Z

docs/user_doc/vic_vsphere_admin/vch_registry.md


-VCHs include a base set of well-known certificates from public CAs. If a registry requires a certificate to authenticate access, and if that registry does not use one of the CAs that the VCH holds, you must provide the CA certificate for that registry to the VCH. If the VCH is running in whitelist mode, you must also add that registry to the whitelist.
+If you register a VCH with a project in vSphere Integrated Containers Management Portal, and if content trust is enabled on that project, vSphere Integrated Containers automatically modifies the whitelist settings on the VCH in order to enforce content trust. 


I've already commented about content trust making a VCH whitelist more restrictive, but not being able to make it more permissive. This paragraph, or the section above about the interaction of whitelists and registries, seem like they might need a reference to that behaviour.

There is a link to the updated section in the bullets below this sentence.

hickeng · 2018-06-29T01:02:41Z

docs/user_doc/vic_vsphere_admin/vch_registry.md

+
+- Deploy a VCH with whitelist mode enabled, and with one or more registries in the whitelist.
+- Register the VCH with a project in vSphere Integrated Containers Management Portal.
+- If content trust is enabled on the project, or if you subsequently enable content trust after adding the VCH to the project, vSphere Integrated Containers automatically overrides the whitelist that you configured on the VCH.


vSphere Integrated Containers automatically restricts the whitelist that you configured on the VCH

Fixed in b3c0052.

hickeng · 2018-06-29T01:10:32Z

docs/user_doc/vic_vsphere_admin/vch_registry.md

+- The VCH whitelist now includes only the registry instance that is running in the vSphere Integrated Containers appliance. 
+- The VCH can now only pull signed and verified images from the registry instance that is running in the vSphere Integrated Containers appliance that includes the project.
+
+If you subsequently disable content trust on a project, or remove a VCH from a project that implements content trust, the VCH reverts to the whitelist configuration with which it was deployed.


If you delete the VIC appliance without removing the VCH the VCH remains paired with that non-existent project and will require a vic-machine configure (I hope a configure is sufficient - I've not tried it). This was done so an attacker cannot use a DoS against the appliance to remove the content trust config applied to the VCH.

Added in b3c0052.

stuclem · 2018-06-29T08:54:23Z

I don't think that we need another round of review. Merging. Thanks @zjs and @hickeng !

Documented how content trust affects VCHs

4208170

stuclem requested review from zjs and hickeng June 26, 2018 10:19

vmwclabot added the cla-not-required label Jun 26, 2018

Clarified that you should not add the same VCH in multiple projects.

22f6ce7

stuclem requested a review from pdaigle June 26, 2018 14:20

zjs reviewed Jun 26, 2018

View reviewed changes

zjs approved these changes Jun 27, 2018

View reviewed changes

hickeng reviewed Jun 29, 2018

View reviewed changes

stuclem mentioned this pull request Jun 29, 2018

Document how to mix signed and unsigned images in a project #1863

Closed

Comments from George

b3c0052

stuclem merged commit 6b1b1d8 into vmware:master Jun 29, 2018

stuclem deleted the contenttrust-vch branch June 29, 2018 08:54

stuclem mentioned this pull request Jun 29, 2018

Document whether adding the same VCH to multiple projects is supported or not #1659

Closed

stuclem added the area/pub Published documentation for end-users label Sep 18, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Documented how content trust affects VCHs #1850

Documented how content trust affects VCHs #1850

stuclem commented Jun 26, 2018

stuclem commented Jun 26, 2018

zjs left a comment

zjs Jun 26, 2018

stuclem Jun 27, 2018

stuclem Jun 27, 2018

stuclem Jun 27, 2018

hickeng Jun 29, 2018

stuclem Jun 29, 2018

stuclem commented Jun 27, 2018

hickeng Jun 29, 2018

hickeng Jun 29, 2018

stuclem Jun 29, 2018

hickeng Jun 29, 2018

stuclem Jun 29, 2018

hickeng Jun 29, 2018

stuclem Jun 29, 2018

hickeng Jun 29, 2018

stuclem Jun 29, 2018

hickeng Jun 29, 2018

stuclem Jun 29, 2018

hickeng Jun 29, 2018

stuclem Jun 29, 2018

hickeng Jun 29, 2018

stuclem Jun 29, 2018

hickeng Jun 29, 2018

stuclem Jun 29, 2018

stuclem commented Jun 29, 2018


		vSphere Integrated Containers Registry provides a Docker Notary server that allows you to implement content trust by signing and verifying the images in the registry.

		Content trust is enabled at the project level. If content trust is enabled on a project, users can only push and pull images to and from that project if they are signed and verified by the Notary server.


		Content trust is enabled at the project level. If content trust is enabled on a project, users can only push and pull images to and from that project if they are signed and verified by the Notary server.

		Enabling content trust on a project automatically modifies the registry whitelist settings of any virtual container hosts (VCHs) that are registered with the project. Consequently, when content trust is enabled, the VCHs in the project can only pull signed and verified images from the registry instance that is running in the vSphere Integrated Containers appliance.


		Enabling content trust on a project automatically modifies the registry whitelist settings of any virtual container hosts (VCHs) that are registered with the project. Consequently, when content trust is enabled, the VCHs in the project can only pull signed and verified images from the registry instance that is running in the vSphere Integrated Containers appliance.

		If you enable content trust on a project, the developers who work on that project must configure their Docker clients so that they can connect to the Notary server running in vSphere Integrated Containers Registry.


		NOTE: vSphere Integrated Containers Management Portal allows you to provision containers from the registries that are included in the lists of global registries that the Management Portal Administrator configures, or project registries that the DevOps administrator configures. However, if the vSphere administrator deployed a VCH with whitelist mode enabled, and if the whitelist on the VCH is more restrictive than the global and project registry lists, you can only provision containers from the registries that the VCH permits in its whitelist, even if the VCH is included in a project that permits other registries. For more information, see [VCH Whitelists and Registry Lists in vSphere Integrated Containers Management Portal](../vic_vsphere_admin/vch_registry.md#vch-whitelist-mp) in vSphere Integrated Containers for vSphere Administrators.
		IMPORTANT: vSphere Integrated Containers Management Portal allows you to provision containers from the registries that are included in the lists of global registries that the Management Portal Administrator configures, or from project registry lists that the DevOps administrator configures. However, if the vSphere administrator deployed a VCH with whitelist mode enabled, and if the whitelist on the VCH is more restrictive than the global and project registry lists, you can only provision containers from the registries that the VCH permits in its whitelist, even if the VCH is included in a project that permits other registries. Also, if you add a VCH to a project on which content trust is enabled, vSphere Integrated Containers updates the whitelist configuration of the VCH. As a consequence, you should only add a given VCH to one project at a time. Adding the same VCH to multiple projects can lead to conflicts if the registry lists and content trust setttings are different in the different projects.


		VCHs include a base set of well-known certificates from public CAs. If a registry requires a certificate to authenticate access, and if that registry does not use one of the CAs that the VCH holds, you must provide the CA certificate for that registry to the VCH. If the VCH is running in whitelist mode, you must also add that registry to the whitelist.
		If you register a VCH with a project in vSphere Integrated Containers Management Portal, and if content trust is enabled on that project, vSphere Integrated Containers automatically modifies the whitelist settings on the VCH in order to enforce content trust.

Documented how content trust affects VCHs #1850

Documented how content trust affects VCHs #1850

Conversation

stuclem commented Jun 26, 2018

stuclem commented Jun 26, 2018

zjs left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stuclem commented Jun 27, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stuclem commented Jun 29, 2018