[discuss] duffle.json vs bundle.json

[isurulucky]

Currently, issuing a duffle create <name> will create a duffle.json, which has several differences between the bundle.json in the original CNAB specification. One such difference I noted is invocationImages element is an object with an abstraction for cnab type in the duffle.json, as shown below:

"invocationImages": {
    "cnab": {
        "name": "cnab",
        "builder": "docker",
        "configuration": {
            "registry": "deislabs"
        }
    }
}

while in the bundle.json its an array of invocation images:

"invocationImages": [
    {
    "imageType": "docker",
    "image": "cnab/helloworld:latest"
  }
]

What is the rationale behind this difference/having an abstraction? I would have thought since duffle is an implementation of the CNAB spec, it can directly output a bundle identifier which exactly matches bundle.json defined in the CNAB spec.

[bacongobbler]

A bundle.json is the input file for the build system (duffle build) that will build a CNAB’s bundle.json. The invocationImages field in duffle.json has a few extra fields, but I do agree that it is confusing how similar duffle.json and bundle.json are.

[bacongobbler]

IIRC there was a discussion in an earlier thread to consolidate the two, though I can’t seem to find that right now.

[radu-matei]

As @bacongobbler pointed out, duffle.json is the input file for Duffle, which will read the info in this file when constructing the bundle.

To specifically address your comment on matching the schema for the invocationImages object in the two files, the main reason for the differences is that the duffle.json file is input for the build system - which might be able to build multiple invocation image types - the default (and what we see as most used) are OCI invocation images (which, if we only consider OCI images, on top of the bundle.json schema for invocationImages need at least a registry property that instructs the OCI builder on how to tag / push the image). But the current builder architecture allows for multiple builders (VM image builders, for example) - and depending on their specific implementations, require different types of properties - hence the InvocationImage type from the manifest contains a Configuration property which can contain any arbitrary info, and each builder extracts whatever information they need.

In an ideal world, any user working with a bundle should be shielded from knowing anything about the CNAB schema (unless they really want / need to. We also don't want users to hand-craft or edit bundle files, as it invalidates a signature), and interact with some high level tooling that understands the CNAB schema - and as of writing this comment, Docker App is adding support for outputting bundles, and there is another declarative tool for building bundles, Porter. While both of these are in early stage, they show how separate tools can implement the spec to output bundles, each of them with their specific way defining input data for building the bundle.

That being said, we understand the confusion generated by having a duffle.json file when initializing the new bundle project, and we're surely open for suggestions on improving the experience.

[isurulucky]

Thanks for the comments @bacongobbler and @radu-matei. Got the point that duffle can build multiple invocation images from a single build. De-coupling the CNAB spec from the user does make sense, however although not exactly the same, duffle.json is quite similar to bundle.json. If its possible to provide the user with a rich experience of creating bundle using the duffle tool, where user does not need to go and re-edit duffle.json by hand, that would improve the user experience and completely hide the spec details from the user in my opinion.

[radu-matei]

If we're talking about the process of authoring bundles using Duffle, would snippets and helper commands in the Duffle VS Code extension help?

Another idea I've been giving a little thought would be helper commands in Duffle itself - something along the lines of:

• duffle bundle add <invocation-image/credential/parameter/image> - this updates the duffle.json file so the user doesn't manually edit it (keep in mind this has not been given much thought, and I'm not sure that bundle is the right subcommand for this).
• duffle build - this builds the bundle and outputs a signed bundle in the local store (by default, or in a local directory, if needed).

Would a combination of these make the experience better?

Thanks!

[jayunit100]

also i think a bundle.json is required for export , which is the biggest value add that I feel like duffle has at the moment? But im very new to the framework so i might be missing something.

[jayunit100]

"A bundle.json is the input file for the build system (duffle build) that will build a CNAB’s bundle.json." <- is that a typo @bacongobbler ?

[bacongobbler]

Yes, sorry. I meant duffle.json

[squillace]

I think the critical item here is the point that only Duffle uses a duffle.json file, correct? The spec can note that while CNAB-compliant tools require only a bundle.json file, tools may have other files for their own use. Here, we can note the opposite in docs. :-)

[jayunit100]

okay, yeah that makes sense. I guess as long as we note it in big letters, since duffle looks like the defacto tool , it should be enough to disambiguate that way . Thanks @bacongobbler and @squillace for all your help on this ! BTW my related issue is solved, so its clear to me now...

[squillace]

I think this deserves some clarification in the CNAB spec and in the duffle docs before we close. I'll see about that.

[isurulucky]

Thanks @squillace. Including that duffle tool uses on duffle.json while CNAB spec requires bundle.json seems the correct clarification in this regard, which should go to the docs.

[michelleN]

@jayunit100 yes, a bundle.json is required for export at the moment but I am currently working on removing that requirement and letting people use bundle references instead of the bundle.json file. See more information on #379 . We're also open to feedback on import/export user experience. Thank you.

[astrieanna]

Considering how similar bundle.json and duffle.json are, I'm not sure why you want bundle authors to use duffle.json. Why not have a bundle.json that's hand-edited and a separate config file for building the invocation image with duffle?

It seems like having the small amount of extra config in a separate file would be less confusing than copying literally the entire bundle.json into duffle.json. If users hand-edit duffle.json, then no details about the spec from being hidden from them. If helper commands are used to avoid hand-editing the duffle.json, then the underlying file being edited could as easily be the bundle.json.

[carolynvs]

Porter uses its config file to abstract away from how CNAB works under the hood. However, having duffle use bundle.json directly makes sense since it is the reference implementation. Unless we are going to take a very different direction with duffle's UX, abstracting isn't really the right thing to be doing.

Working directly with the bundle.json is less confusing than editing the duffle.json and also having the bundle.json as an output to deal with too. It improves people's understanding of the CNAB spec and also prevents us from papering over problems with the format in the duffle.json file format. 😀