Question: nullable and enum

[left-rite]

Hey,

I am hoping to get clarification on the following:

When the nullable and enum keywords are used together, which takes precedence?

i.e.

type: string
nullable: true
enum:
  - one
  - two

propertyA = null

When propertyA is validated against the definition. Should the result be:

1. true, since the it's nullable
2. false, because null is not part of the enum
3. something else?

[tedepstein]

@left-rite, your first answer is correct:

When propertyA is validated against the definition. Should the result be:

1. true, since it's nullable.

I think of nullable is a shortcut for an anyOf applicator with two subschemas, like this:

anyOf:
- type: "null"
- {schemaObject}

... where {schemaObject} is a placeholder for the schema you've defined, excluding the nullable keyword.

(Please note that type: "null" is supported in JSON Schema, but not in OpenAPI. OpenAPI uses the nullable keyword for this purpose.)

Using this theoretical translation, your example would translate into this JSON Schema:

anyOf:
- type: "null"
- type: string
  enum:
  - one
  - two  

This is consistent with the way nullable works in other cases. enum is a validation constraint, just like type, pattern, minItems, etc. In all cases, non-null values must validate against these constraints. But if nullable is true, then null is allowed as a value regardless of any other constraints.

[epoberezkin]

The docs say here that "enum" behaves as defined in JSON Schema spec. "enum" being affected by "nullable" is not consistent with JSON Schema spec.

[tedepstein]

@epoberezkin , I would say that all assertions in JSON Schema are affected by nullable. If the Schema Object is nullable, the value can non-null and conformant to the schema; or it can be null, in which case none of the schema constraints apply.

[tedepstein]

@epoberezkin , looking more closely at the JSON Schema spec:

3.2.1. Assertions and Instance Primitive Types

Most validation assertions only constrain values within a certain primitive type. When the type of the instance is not of the type targeted by the keyword, the instance is considered to conform to the assertion.

For example, the "maxLength" keyword will only restrict certain strings (that are too long) from being valid. If the instance is a number, boolean, null, array, or object, then it is valid against this assertion.

Does enum apply to null values?

6.1.2. enum

The value of this keyword MUST be an array. This array SHOULD have at least one element. Elements in the array SHOULD be unique.

An instance validates successfully against this keyword if its value is equal to one of the elements in this keyword's array value.

Elements in the array might be of any value, including null.

JSON Schema categorizes its keywords as assertions, annotations, and applicators. A few keywords may function as more than one of these categories.

I think nullable is problematic because it doesn't fit into any of these categories. Where assertions constrain the allowed value space, nullable has the effect of expanding the value space to include null, which would otherwise be disallowed.

You can easily see how this could cause a problem with allOf inheritance, where adding nullable to a subtype violates the constraints of the supertype:

    Bird:
      type: object
      properties:
        order:
          type: string
        genus:
          type: string
        species:
          type: string
        flightSpeed:
          type: number
          
    FlightlessBird:
      allOf:
      - $ref: "#/components/schemas/Bird"
      properties:
        flightSpeed:
          type: number
          nullable: true

But JSON Schema allows all kinds of contradictory schemas. So using it in a subtype would not override the constraints of the supertype. An object with flightSpeed: null would still fail validation against FlightlessBird, because it's prohibited in the supertype.

I'm inclined to agree that the OpenAPI spec should be clarified here, but we would need to figure out how best to do it.

[epoberezkin]

OpenAPI spec should be clarified here, but we would need to figure out how best to do it.

2 options:

1. exclude "enum" from the list of keywords that follow JSON Schema spec and to re-define "enum" to allow null value with "nullable" even if null is not explicitly listed in "enum".
2. require that "enum" explicitly lists null to allow it.

[tedepstein]

@epoberezkin , I like your suggested options.

They also seem to correspond to two different translations of nullable:

---

1. nullable: true --> anyOf: [s, type: 'null']

Where s is the full set of assertions (excluding nullable) defined in the schema. This is the interpretation I described here, where schema assertions apply only to non-null values.

So enum, like other assertions, wouldn't apply here. This has the same effect as redefining enum in OpenAPI to implicitly allow null values when nullable is true.

---

2. nullable: true --> type: [t, 'null']

Where t is the type specified in the schema.

Or, if type is not specified in the schema, t is the list of all allowed, non-null types. In that case, nullable: true translates to type: [integer, number, string, boolean, object, array, null]

This is another plausible translation of nullable to JSON Schema. Semantically, it's different from the first one because all of the other schema assertions, including enum, would still apply. In that case, an enum would disallow null values unless it includes null explicitly as one of its enumerated values.

---

I would rather see the spec clarification done in this form, as a translation of nullable: true to one of these two JSON Schema constructs. Though it's not as straightforward as what you proposed, it accomplishes a few useful things:

1. Reconciles OpenAPI with JSON Schema, providing an unambiguous translation so we can apply standard JSON Schema validation semantics.
2. Allows for automated translation so we can make use of actual implementations of JSON Schema validation.
3. Gets us out of the danger zone wherein nullable is an entirely new category of keyword that expands the value space, rather than constraining or annotating.
4. Clarifies the interactions between nullable and other schema assertions, aside from enum, that are also type-agnostic. Right now it looks like const is the only other type-agnostic assertion, and it's not supported in OpenAPI. But later revisions of the spec could add support for const, or other type-agnostic assertions introduced either directly by OpenAPI or via JSON Schema.

[tedepstein]

Note the related discussion in #1389, which proposes replacing nullable with a 2-element type array, where one element is the name of a supported type and the other element is "null". This is a special case of JSON Schema's type arrays, so it aligns better with JSON schema. And it matches translation 2, in my comment above.

So I started out saying that translation 1 was the correct way to interpret nullable, but it seems like there's more of a consensus around translation 2.

And in that case, the enum constraint would still apply, and would prohibit null values unless the enum explicitly includes null.

[tedepstein]

Please note that a proposed clarification is now under review, hopefully to be included in a v3.0.3 patch release of the OpenAPI spec:

Proposal: Clarify Semantics of nullable in OpenAPI 3.0

[handrews]

@tedepstein @MikeRalphson @webron since the clarification did make it into v3.0.3 can this be closed now?

[webron]

Yup!

[jdegre]

I see this issue is closed, but maybe I can still get clarification on the recommended approach to define in OpenAPI 3.0.x a data type where the set of allowed values consist on an enum (e.g. of type string) and also the null value. Since nullable: true seems to be not correct anymore, how should it be defined?

Would this approach be ok, for example?

NullableEnum:
  anyOf:
    - type: string
      enum:
        - one
        - two
    - enum: [ null ]

[tedepstein]

@jdegre, that would work. But this also works:

NullableEnum:
  type: string
  nullable: true
  enum:
    - one
    - two
    - null

The combination of type: string with nullable: true is equivalent to the type array type: [string, "null"] in JSON Schema. And in fact, the current plan is to support full JSON Schema in OpenAPI 3.1. So in OpenAPI 3.1, this will be allowed:

NullableEnum:
  type: [string, "null"]
  enum:
    - one
    - two
    - null

[jdegre]

thanks a lot for the fast reply @tedepstein!

[cgfarmer4]

For posterity, we had an issue using the python generator with this format:

NestedNullableEnum:
  type: string
  nullable: true
  enum:
    - one
    - two
    - null

Could not generate model 'NestedNullableEnum'
	at org.openapitools.codegen.DefaultGenerator.generateModels(DefaultGenerator.java:542)
	at org.openapitools.codegen.DefaultGenerator.generate(DefaultGenerator.java:879)
	at org.openapitools.codegen.cmd.Generate.execute(Generate.java:432)
	at org.openapitools.codegen.cmd.OpenApiGeneratorCommand.run(OpenApiGeneratorCommand.java:32)
	at org.openapitools.codegen.OpenAPIGenerator.main(OpenAPIGenerator.java:66)
Caused by: java.lang.NullPointerException: Null context for variable 'isString' on line 141

But this one worked fine:

NestedNullableEnum:
  anyOf:
    - type: string
      enum:
        - one
        - two
    - enum: [ null ]

[spacether]

@tedepstein was https://github.com/OAI/OpenAPI-Specification/blob/master/proposals/003_Clarify-Nullable.md approved?
If so can it be described as approved and list that it was included in 3.0.3?

[richardwhiuk]

Has anyone given any thought to such constructs as:

nullable: false
enum:
  - one
  - two
  - null

[spacether]

@richardwhiuk my interpretation of your spec per v3.0.3 is that nullable false has no impact on types. Type null is allowed as oneof the any types and that all three enum values should be accepted

[handrews]

@richardklose enum only validates values in its list, the nullable is irrelevant.

During the lead-up to 3.0.3 and 3.1.0, we had a lot of discussion over the problems introduced if nullable is allowed to contradict the restrictions imposed by other keywords in an uncontrolled way. This is why the wording around nullable in OAS 3.0.3 changed to be specifically a modification of type, and nothing else:

A true value adds "null" to the allowed type specified by the type keyword, only if type is explicitly defined within the same Schema Object. Other Schema Object constraints retain their defined behavior, and therefore may disallow the use of null as a value. A false value leaves the specified or default type unmodified. The default value is false.

So enum in your example does not include null in is list of values, therefore nullable has no real effect (even if type were also present, adding the "null" type doesn't matter because null is not in the enum).

[richardwhiuk]

The example I gave has null in the enum, but nullable false?

[handrews]

@richardwhiuk if you want to forbid null while using an enum, you need to either remove null from the enum or set type: string so that the nullable: false takes effect. Without type, nullable has no effect (in OAS 3.0.3, and it may not be implemented consistently since it was ambiguous in 3.0.0-3.0.2). This is part of why nullable is gone in OAS 3.1.

[tedepstein]

@spacether ,

@tedepstein was https://github.com/OAI/OpenAPI-Specification/blob/master/proposals/003_Clarify-Nullable.md approved?
If so can it be described as approved and list that it was included in 3.0.3?

Thanks for the reminder. I have opened PR #2529 to update the proposal as you've suggested.

@richardwhiuk,

The example I gave has null in the enum, but nullable false?

This excerpt from the proposal may help to clarify why your schema permits null even though it specifies nullable: false:

Detailed design

According to the above specification, nullable only operates within a narrow scope, wherein its translation to JSON Schema is straightforward:

• nullable is only meaningful if its value is true.

• nullable: true is only meaningful in combination with a type assertion specified in the same Schema Object. nullable acts as a type modifier, allowing null in addition to the specified type.

• nullable: true operates within a single Schema Object. It does not "override" or otherwise compete with supertype or subtype schemas defined with allOf or other applicators. It cannot be directly "inherited" through those applicators, and it cannot be applied to an inherited type constraint.

This also solves the issues of alignment with JSON Schema:

• Since type is a constraint, JSON Schema's constraint-based processing model is fully applicable. Interactions between type and other constraining assertions and applicators are unambiguous, with each constraint having independent veto power.

• It is now clear that nullable: false, whether explicit or by default, does not prohibit null values. Consistent with JSON Schema, an empty object allows all values, including null.

[EvanCarroll]

Boy the wording here is especially confusing and overly verbose.

Can you just show with code what is allowed and not allowed.

nullable is only meaningful if its value is true.

What value? The type that contains the union of enum and null, or a null type constructor in an enum?

IE, how do we model

enum T { A, B, NULL };
foo: T;

And

enum T { A, B };
foo: Option<T>;  /// (or Maybe<T> _whatever)_

What would the examples on an OAPI3 schema look like for both of these? Would this be valid,

                          ecdsa_curve_name:__
                            description: |-
                              The ECDSA curve that the certificate's key uses.
                              * `prime256v1`
                              * `secp384r1`
                              * `null` — The certificate's key is **not** an ECDSA key.
                            enum:
                            - prime256v1
                            - secp384r1
                            example: ~
                            nullable: true
                            type: string

[tedepstein]

nullable is only meaningful if its value is true.

What value? The type that contains the union of enum and null, or a null type constructor in an enum?

The value of the nullable property itself.

nullable: true, meaning nullable with a value of true, is meaningful in certain contexts. In those contexts, it can cause null values to be allowed where they would otherwise be disallowed.

nullable: false is meaningless, meaning it never has any effect on the validity of null values, nor any effect on anything else.

[spacether]

@EvanCarroll you need to add the null value into the enum options.
nullable: True only allows in string values and the value null.
Then the value is checked against the constrained enum values. Per your sample null isn't in the enum, so the value null would not be allowed. Your example schema is valid and validation will only succeed with payload values of 'prime256v1' and 'secp384r1'

[EvanCarroll]

@EvanCarroll you need to add the null value into the enum options.
nullable: True only allows in string values and the value null.
Then the value is checked against the constrained enum values. Per your sample null isn't in the enum, so the value null would not be allowed. Your example schema is valid and validation will only succeed with payload values of 'prime256v1' and 'secp384r1'

Thanks a ton! That's far clearer than the RFC and conversation.

[EvanCarroll]

@spacether as a side note. What is the rule of nullable: True in enums then? Ie, can you craft an example with nullable: falseand nullable: true

[EvanCarroll]

@tedepstein

What value? The type that contains the union of enum and null, or a null type constructor in an enum? The value of the nullable property itself. nullable: true, meaning nullable with a value of true, is meaningful in certain contexts. In those contexts, it can cause null values to be allowed where they would otherwise be disallowed. nullable: false is meaningless, meaning it never has any effect on the validity of null values, nor any effect on anything else.

I don't think you're seeing the confusion here.. The term "value" there in the context of a type system is remarkably confusing.

enum Foo { A , B, Null }

What's the value in the above? The right answer, afaik, is there is no value. There is a type. That type has two value constructors A, and B, and Null. Values would like this:

a = Foo::A
b = Foo::B
c = Foo::Null

We can also model this,

enum Option<T> {
  None,
  Maybe(T)

}

Again there is no value there.

So now we have two possible interpretations of Null one is as a value constructor on the enum Foo. The other is as a value constructor on the enum Option<T>. I may be wrong, but I know of both from Haskell and Rust as value constructors.

So I need to know which of these the "nullable: True` is enabling. Is it,

1. Adding a value constructor to the enum,
2. Making it such that the enum is optional and can be null, (most logical)
3. Neither, and does nothing if the type is enum.

Thus far @spacether's answer clearly tells me that it's not doing 1, and I have to do that. And because that's not how I'd prefer to model this, I presume 2 isn't an option. So the question then is what does nullable: True do in the "context" of a parameter that is an enum . And if it does nothing in that context, shouldn't the very construct of nullable: True be an error in the schema (as should be nullable: False anywhere)?

[handrews]

@EvanCarroll you realize OAS is not a C++ code generation specification, right? Anything about mapping OAS to C++ code is going to be specific to your tooling. OAS says what will and won't validate as a JSON/YAML/etc. instance. That doesn't always map cleanly to every programming language, and it's up to code generators to decide what to do with those scenarios.

nullable: True, in the presence of a type field, means that a JSON null is valid in addition to whatever type (plus constraints from other keywords including enum) the schema allows That's it. That's all OAS says. If a value fails against the enum constraint, then it doesn't matter what the other keywords in the schema do, it's already failed. OAS does not say anything about constructors, or the subtleties of null values vs typing systems in different languages. That is is outside of the scope of the spec.

[tedepstein]

@EvanCarroll , I think you will find it easier to understand if you keep in mind that OpenAPI is entirely declarative. There are no constructors or other imperative statements.

• The JSON specification lays out a simple type system and a syntax for representing data in that type system.
• YAML is another general-purpose declarative language that can be used as a DSL for JSON. That's how OpenAPI uses it.
• JSON Schema provides a validation language (not a type system, and not a programming language) for JSON and JSON-like data. And a JSON schema itself usually takes the form of a JSON or YAML document.
• OpenAPI is a description language for APIs, and it uses JSON Schema to describe and constrain the data that can be passed through the API.

In all of these declarative languages, maps, consisting of key-value pairs, are ubiquitous. And in that context, all of those language specs use the term "value" extensively to refer to the value in a key-value pair. These technologies are the world we live in. This is the context for most of these discussions.

nullable: true is a key-value pair, where nullable is the key and true is the value. That's all I meant; no reason to get hung up on this. I know this doesn't directly answer your specific questions about nullable, but I'm hoping it helps get you past that sticking point so the answers, comments, and resources given here will make more sense to you.