Define end-to-end test for version endpoint [specific ci=Group23-Vic-Machine-Service] #6475

zjs · 2017-09-28T22:02:55Z

Introduce a single-test robot suite for the version endpoint as well as a utility resource for use by this suite and others within the group.

jzt · 2017-09-28T23:27:43Z

tests/test-cases/Group23-VIC-Machine-Service/23-00-Util.robot

+    ${auth}=    Evaluate    base64.b64encode("%{TEST_USERNAME}:%{TEST_PASSWORD}")    modules=base64
+    ${RC}  ${OUTPUT}=    Run And Return Rc And Output    curl -s -w "\n\%{http_code}\n" -X GET "http://127.0.0.1:${HTTP_PORT}/container/target/%{TEST_URL}/${PATH}?thumbprint=%{TEST_THUMBPRINT}" -H "Accept: application/json" -H "Authorization: Basic ${auth}"
+    ${STATUS}=    Get Line    ${OUTPUT}    -1
+    Set Test Variable    ${RC}


I haven't seen this pattern before, but I think I like it. It makes the tests very clean.

mdharamadas1 · 2017-09-29T13:55:59Z

tests/test-cases/Group23-VIC-Machine-Service/23-00-Util.robot

+    ${auth}=    Evaluate    base64.b64encode("%{TEST_USERNAME}:%{TEST_PASSWORD}")    modules=base64
+    ${RC}  ${OUTPUT}=    Run And Return Rc And Output    curl -s -w "\n\%{http_code}\n" -X GET "http://127.0.0.1:${HTTP_PORT}/container/target/%{TEST_URL}/${PATH}?thumbprint=%{TEST_THUMBPRINT}" -H "Accept: application/json" -H "Authorization: Basic ${auth}"
+    ${STATUS}=    Get Line    ${OUTPUT}    -1
+    Set Test Variable    ${RC}


I wouldn't suggest having these keywords in a separate file as I think it will be run as test suite as its under test suite dir but @mhagen-vmware can confirm that. Also, I don't like the corresponding .md file.

I wouldn't suggest having these keywords in a separate file as I think it will be run as test suite as its under test suite dir but @mhagen-vmware can confirm that.

If it gets run as a suite, it seems like it will get run as a suite with zero tests. That isn't a problem locally. Is it a problem for our CI infrastructure?

(Replying to @mhagen-vmware here, to keep related conversation in a single thread.)

this should be a resource file within our resources folder and needs to be added into resources/Util.robot please

These keywords aren't intended to be used by other test groups. Defining them in the resources folder and adding that file to resources/Util.robot essentially adds them to a global namespace.

Do we have a better pattern for sharing keywords between suites within a single test group?

Is there a functional problem with this pattern, or is the complaint only that it's stylistically different than what we've done in the past?

I don't like the corresponding .md file.

I don't like it either, but was attempting to following the formatting conventions. I'd rather use that file as documentation for these keywords, but I think I'd need to format it differently. Are we okay with that?

Is it a problem for our CI infrastructure?

No, but that pattern (having util files under tests) doesn't seem right to me.

Are we okay with that?

I would suggest removing it for now as we don't have any documentation for existing keywords.

No, but that pattern (having util files under tests) doesn't seem right to me.

I moved the resource file so that it's under resources, and named it in a way that makes it clear it's intended for use within this test group.

I would suggest removing it for now as we don't have any documentation for existing keywords.

I removed the corresponding md file.

mhagen-vmware

yea, this is really not using any of our existing patterns.. this should be a resource file within our resources folder and needs to be added into resources/Util.robot please

mdharamadas1

few suggestions which i think will make this pattern better.

mdharamadas1 · 2017-10-02T16:13:07Z

tests/resources/Group23-VIC-Machine-Service-Util.robot

@@ -0,0 +1,59 @@
+# Copyright 2017 VMware, Inc. All Rights Reserved.


Can we rename this file to something more general, may be like VIC-Machine-Service-Util.robot? We may use it in other test suites as well.

I don't want to give the file a more general name because I don't want these keywords used in other test groups.

Our current process allows for only two types of keywords: suite-scoped and global. This file follows a different pattern: group-scoped.

Defining all shared keywords in the global scope carries many of the same drawbacks as defining variables in the global scope: identifying relevant keywords becomes harder, determining where keywords are used becomes harder, safely changing the behavior of keywords becomes harder, concisely and accurately naming keywords becomes harder, etc.

We'll iterate on this design as we learn more about this pattern, but I think this is a reasonable starting point.

mdharamadas1 · 2017-10-02T16:15:00Z

tests/resources/Group23-VIC-Machine-Service-Util.robot

+    Set Test Variable    ${STATUS}
+
+
+Verify Return Code


I would put all these verify keywords in a separate util file, may be, called Verify-Util.robot.

Given the close dependency between the way the Get (and eventually Post/Put/Delete) methods store results and the way the Verify methods make assertions, I think it's important to keep the keywords in the same file; using the Verify keywords without the Get keywords wouldn't make sense.

mdharamadas1 · 2017-10-02T16:16:51Z

tests/resources/Group23-VIC-Machine-Service-Util.robot

+    Start Process    ./bin/vic-machine-server --port ${HTTP_PORT} --scheme http    shell=True    cwd=/go/src/github.com/vmware/vic
+
+
+Get Path


As this is in global space now, can we rename this keyword to little more specific?

Placing these keywords in the global scope would, as you suggest, require providing more specific names for the keywords. However, within the scope of this test group I think the keyword names are clear.

mdharamadas1 · 2017-10-02T16:17:23Z

tests/resources/Group23-VIC-Machine-Service-Util.robot

+    Set Test Variable    ${STATUS}
+
+
+Get Path Under Target


As this is in global space now, can we rename this keyword to little more specific?

mdharamadas1

@zjs I'm fine with iterating this further later, would suggest adding an issue for that.

zjs · 2017-10-04T20:05:06Z

I'm fine with iterating this further later, would suggest adding an issue for that.

Good idea! Filed #6508.

zjs added the team/lifecycle label Sep 28, 2017

zjs self-assigned this Sep 28, 2017

zjs requested review from jzt, mhagen-vmware and mdharamadas1 September 28, 2017 22:02

vmwclabot added the cla-not-required label Sep 28, 2017

zjs force-pushed the feature/vic-machine-service branch from 5511407 to 75bbbc0 Compare September 28, 2017 22:17

zjs force-pushed the topic/vic-machine-service-robot-version branch 2 times, most recently from 1aedfcd to 2c72e45 Compare September 28, 2017 22:40

jzt reviewed Sep 28, 2017

View reviewed changes

jzt approved these changes Sep 28, 2017

View reviewed changes

mdharamadas1 reviewed Sep 29, 2017

View reviewed changes

mhagen-vmware suggested changes Sep 29, 2017

View reviewed changes

zjs force-pushed the topic/vic-machine-service-robot-version branch from 404f76e to 5998081 Compare September 29, 2017 22:29

zjs added this to the Sprint 18 Lifecycle milestone Sep 30, 2017

mdharamadas1 suggested changes Oct 2, 2017

View reviewed changes

zjs changed the title ~~Define end-to-end test for version endpoint [specific ci=23-01-Version]~~ Define end-to-end test for version endpoint [specific ci=Group23-Vic-Machine-Service] Oct 2, 2017

zjs force-pushed the topic/vic-machine-service-robot-version branch 2 times, most recently from 448a3c8 to c2d649a Compare October 3, 2017 18:36

zjs mentioned this pull request Oct 3, 2017

Define end-to-end tests for VCH listing endpoint [specific ci=Group23-Vic-Machine-Service] #6496

Merged

mdharamadas1 approved these changes Oct 4, 2017

View reviewed changes

zjs mentioned this pull request Oct 4, 2017

Refine patterns used for vic-machine service end-to-end tests #6508

Closed

zjs closed this Oct 4, 2017

zjs force-pushed the topic/vic-machine-service-robot-version branch from c2d649a to 45d4f6b Compare October 4, 2017 20:29

zjs deleted the topic/vic-machine-service-robot-version branch October 4, 2017 20:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Define end-to-end test for version endpoint [specific ci=Group23-Vic-Machine-Service] #6475

Define end-to-end test for version endpoint [specific ci=Group23-Vic-Machine-Service] #6475

zjs commented Sep 28, 2017

jzt Sep 28, 2017

mdharamadas1 Sep 29, 2017 •

edited

Loading

zjs Sep 29, 2017

mdharamadas1 Sep 29, 2017

zjs Sep 29, 2017

mhagen-vmware left a comment

mdharamadas1 left a comment

mdharamadas1 Oct 2, 2017

zjs Oct 2, 2017

mdharamadas1 Oct 2, 2017

zjs Oct 2, 2017

mdharamadas1 Oct 2, 2017

zjs Oct 2, 2017

mdharamadas1 Oct 2, 2017

zjs Oct 2, 2017

mdharamadas1 left a comment

zjs commented Oct 4, 2017

		@@ -0,0 +1,59 @@
		# Copyright 2017 VMware, Inc. All Rights Reserved.

		Start Process ./bin/vic-machine-server --port ${HTTP_PORT} --scheme http shell=True cwd=/go/src/github.com/vmware/vic


		Get Path

Define end-to-end test for version endpoint [specific ci=Group23-Vic-Machine-Service] #6475

Define end-to-end test for version endpoint [specific ci=Group23-Vic-Machine-Service] #6475

Conversation

zjs commented Sep 28, 2017

Choose a reason for hiding this comment

mdharamadas1 Sep 29, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mhagen-vmware left a comment

Choose a reason for hiding this comment

mdharamadas1 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mdharamadas1 left a comment

Choose a reason for hiding this comment

zjs commented Oct 4, 2017

mdharamadas1 Sep 29, 2017 •

edited

Loading