Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] update initruntimetest by linzhinan #61

Closed

Conversation

liangchenye
Copy link
Member

This pr is still under development.
Since we have 'state' and 'lifecircle' defined now (comparing to zhinan's previous PR #9)
This PR not only test the main configs (verified by cmd/runtimetest), but also test the state and life circle.

Signed-off-by: linzhinan(zen Lin) <linzhinan@huawei.com>
@liangchenye
Copy link
Member Author

any suggestions on state test and lifecirle test? @zenlinTechnofreak

"strings"

"github.com/Sirupsen/logrus"
"github.com/opencontainers/ocitools/config"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Travis doesn't like this import, and you don't seem to use config.… in unit.go anyway, so you can probably drop this line.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, you dropped the config package in cb986be and just forgot this line.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for correction, I'll change that add post a working version.

@wking
Copy link
Contributor

wking commented May 9, 2016

There was some consensus in #47 about making runtime testing a
first-class command (1 and later). That would be good, but I'm not
sure how to get there. One problem with this Go-based approach is
that it's a lot more work to test command-line invocation
(e.g. testState 2) than it would be in a shell-based test framework
(e.g. 3). So as much as I'd like an easily-installable command for
testing, my current preference is still to exercise the runtime's
command-line API through a command-line test framework 4. RunC's
test suite is likely a good starting point for this, we'd just have to
trim off the out-of-spec components.

Signed-off-by: linzhinan(zen lin) <linzhinan@huawei.com>
@liangchenye liangchenye force-pushed the initruntimetestCmd branch 3 times, most recently from d59e6a4 to 9c08ae3 Compare May 10, 2016 08:56
Signed-off-by: liang chenye <liangchenye@huawei.com>
@liangchenye liangchenye force-pushed the initruntimetestCmd branch 5 times, most recently from 6930018 to ad60254 Compare May 10, 2016 14:00
@liangchenye
Copy link
Member Author

liangchenye commented May 10, 2016

The purpose of using golang in runtime test (or runtime certification) is we need to test more cases.
For example, the state test and lifecircle test are different with cmd/runtime test (I called it main config test in these commits). So the command-line script will become bigger and more complex if we add these. TestUnit is a wrap of a container, once there are more test cases, it is easier to apply and maintain them.

@wking @mrunalp PTAL

@wking
Copy link
Contributor

wking commented May 10, 2016

On Tue, May 10, 2016 at 07:16:54AM -0700, 梁辰晔 (Liang Chenye) wrote:

For example, the state test and lifecircle test are different with
cmd/runtime test (I called it main config test in these commits). So
the command-line script will become bigger and more complex if we
add these.

I'm not recommending a single command-line script, I'm recommending
using a script-based framework (like runC already does 1). The bats
equivalent to TestUnit is [2,3], although the latter is killing the
container instead of just waiting for it to exit. In ccon I'm using
sharness, and tests look like 4. I expect that approach to scale
pretty well, and ‘test_expect_code 1 …’ (e.g. 5) is a very compact
way to check non-zero exit codes.

Signed-off-by: liang chenye <liangchenye@huawei.com>
Signed-off-by: liang chenye <liangchenye@huawei.com>
Signed-off-by: liang chenye <liangchenye@huawei.com>
@wking
Copy link
Contributor

wking commented Jul 26, 2016

It seems like the resistance to using bats, sharness, Python (#98),
etc. is the desire to ship a single binary to cover all ocitools
functionality 1. I've been looking around for a Go analog to
Python's unittest that can be built into the installed binary, but so
far haven't turned up anything useful. It's possible that we could
use Go's testing library and create our own runner with testing.Main
[2](since ‘go test’ expects *_test.go, and those are excluded from
the regular package [3]). But while it seems like it should be
possible, I haven't been able to figure out how to do it.

I'm not excited about writing and maintaining our own Go test harness,
so I'd appreciate if folks who prefer a single compiled binary to a
scriptier Bash / POSIX shell / Python solution could lay out their
reasoning for wanting the binary. Do we really expect folks to want
to run ocitools on systems without Python?

@Mashimiao
Copy link

any updates?

@liangchenye
Copy link
Member Author

Close and split it into small issues/PRs.

@wking wking mentioned this pull request Mar 6, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants