Allow tests to reference the key in messages to users

[savage-alex]

As a rule creator (and a consumer of the report)
I want to receive verbose messages informing me of the location of an issue with its name
So that I can quickly see what went wrong with my design.

(This makes allot more sense when designing on an unresolved definition and exporting a resolved one into spectral.)

Given the following rule:

The consumer is told exactly which parameter has the fault (and the line and ruleset)

Perfect right?

Unfortunately with truthy evaluations for a property it doesn't seem possible to get the key out.
I have tried to include the"hint" {parameter} or {value} in the rule:

Rule using property to pull through:

Results in:

Rule changed to return the value, results in:

Is it possible we can get the key returned as an option? I think it would be really helpful to consumers

[P0lip]

Hey!
Key should be already supported.
Did you try using {property}?

[savage-alex]

@P0lip hey there, see the above extract when using {{property}} i get a response back of [1] example

Can provide you the API definition and rule if it helps?

[P0lip]

Ah, it was said {parameter} in the initial description, so I assumed you simply misspelled it, but now I see it returns [1].example.
If I understand the issue correctly, you'd like to have just example printed out, right?
If not, what would be the ideal output for you?

[savage-alex]

It would be great if it were able to provide the value from another property? such as parameter name?
Given the example below

It would be super useful to report the name of the parameter at fault.

[P0lip]

Agreed. That'd indeed provide a much more meaningful feedback.
I'll try to see what we can do about it.

[savage-alex]

Trying to get to the point that @arno-di-loreto has with the path / operation tests surfacing really nice responses ala https://apihandyman.io/toolbox/spectral/

I have the rules for 400, 401 .... 415 when there is a request body.. But would really level up with the path and operation in the message

[arno-di-loreto]

If the question is “how do I get the path in message “, I use the following conf using path variable on each rule:

message: “{{description}} ({{path}})”

[savage-alex]

Way to save the day and give me loads of JEST tests to change ;-)

Thank you @arno-di-loreto ! Solves one on my problems. What do you think about the top request?

[P0lip]

@savage-alex I put some draft PR over here #1341, which should address your first concern. I'm not quite sure as to its final shape yet, but either way, please let me know whether it meets your requirements.

[savage-alex]

Hey @P0lip, im afraid its a bit beyond me to review. Happy to have a quick MS teams call if your game to take a look at it in the flesh?

[P0lip]

Hey, no worries, you don't need to review it.
I'd just like you to take a look whether the output you desire would be possible to generate once my change is merged.
I can compile a Spectral binary if you want to try it out and don't want to clone the whole Spectral repo.
Does that sound good?

[savage-alex]

Hey, I have the PR running locally. Given the following OAS parameter against an operation:
parameters:
- in: query
name: includeInactive
description: false by default. If true, includes inactive tenants in the response
schema:
type: boolean
required: false

there is no example. My rule currently sits at:

adv-query-parameters-example-exists:
description: All query parameters SHOULD have an example.
severity: error
message: 'All query parameters SHOULD have an example.'
given: "$..parameters[?(@.in == 'query')]"
then:
field: example
function: truthy

To retrieve the property name:includeInactive in the message as that is where the fault is, what would the message need to have?

[P0lip]

@savage-alex
Hey, this would be

rules:
  adv-query-parameters-example-exists:
    description: All query parameters SHOULD have an example.
    severity: error
    message: 'Query parameter #{{value.name}} SHOULD have an example.'
    given: "$..parameters[?(@.in == 'query')]"
    then:
      field: example
      function: truthy

[savage-alex]

Looks perfect to me! (After I had remembered I had to actually build spectral and use the new binary. Such a n00b I am.

Well done and thank you on the fix! Happy to discuss further on if it will make the cut or not.

[savage-alex]

@P0lip
I have done some more testing this morning and I think this introduced a bug:

• If you create an example property but fail to give it a value, you get an error
  

API definition code:

You can see where the baseline spectral which is in my VScode has noticed there is a value missing and is highlighting that.

Linting the same file with the baseline finds the value missing and reports the normal error from truthy.

[P0lip]

@savage-alex could you send me the rule definition?

[savage-alex]

Sure email or have you got slack or teams so can share there?
Can give you the API def too that way. Rather than filling up GitHub in the clear.

[P0lip]

@savage-alex
We have Stoplight Slack channel, but I'm also available on APIs You Won't Hate.
You can also send an email to jakub(at)stoplight.io.
Seems like email might be the easiest way?