Add messages implementation for python #165

elchupanebrej · 2023-07-18T18:08:37Z

🤔 What's changed?

Add python implementation

🏷️ What kind of change is this?

⚡ New feature (non-breaking change which adds new behaviour)

📋 Checklist:

I agree to respect and uphold the Cucumber Community Code of Conduct
I've changed the behaviour of the code
- I have added/updated tests to cover my changes.
Users should know about my change
- I have added an entry to the "Unreleased" section of the CHANGELOG, linking to this pull request.

This text was originally generated from a template, then edited by hand. You can modify the template here.

elchupanebrej · 2023-07-18T18:11:58Z

This address to #162

mpkorstanje

At a glance this doesn't follow the pattern used by the other language implementations in quite a few ways. Please follow up the directions from #162 around code generation.

I also don't understand the purpose of the samples directory.

elchupanebrej · 2023-07-19T12:50:14Z

@mpkorstanje

For Python exists a tool that allows generating Pydantic models directly from json schema https://github.com/koxudaxi/datamodel-code-generator - So this allows not including an extra layer with templating. If you insist - I'll rewrite this by that approach.
Samples are taken from gherkin repository to validate if serialization/deserialization works well. Adding external data to a python package is always an egg-chicken problem. I don't like to add external files by makefiles or any kind of scripts because they are always platform dependent. If another approach is used in cucumber - please let me know, and I'll adapt this PR

mpkorstanje · 2023-07-19T15:34:16Z

For Python exists a tool that allows generating Pydantic models directly from json schema

You can use Pydantic if you can make it fit into the make clean-all generate-all workflow. Though I suspect your manual edits might pose a problem.

Samples are taken from gherkin repository to validate if serialization/deserialization works well.

Consider narrowing this down to a few representative examples. Currently it is hard to see the forest for the trees.

luke-hill

If you're going to copy lots of the cck it would be better to fetch the data using some form of call rather than C+P as this is currently being rapidly updated

python/pyproject.toml

.github/workflows/test-python.yml

python/RELEASING.md

mpkorstanje · 2024-01-04T12:55:15Z

@elchupanebrej

Samples are used in tests. More complex tests could exist. I insist to include them for now

What purpose do these tests serve? They'll be a hassle to update if/when the schema changes.

luke-hill · 2024-04-04T14:46:02Z

Hi @elchupanebrej - Just checking in to see where you're up to with this. Is this something you're still working on?

elchupanebrej · 2024-09-04T17:13:54Z

Hi @elchupanebrej - Just checking in to see where you're up to with this. Is this something you're still working on?

Hi @luke-hill, sorry for the long response, hadn't time to work on the project. I'll try to create another merge request that will conform to the building process.

elchupanebrej · 2024-09-04T20:59:59Z

The PR was updated with Makefile. Model is stable, so generated code is totally same to version, which was generated at first try

@mpkorstanje I kindly ask you to review the code and take a release part. I didn't get into all deps&relations between release tools.

mpkorstanje · 2024-09-04T21:18:59Z

python/tests/test_model_load.py

+def compatibility_kit_repo(tmpdir):
+    repo_path = Path(tmpdir) / "compatibility-kit"
+    repo = Repo.clone_from(
+        "https://github.com/cucumber/compatibility-kit.git",


Messages should not use the compatibility kit as this creates a circular dependency. Rather you'll want to write some targeted tests for serialization and deserialisation.

The Java implementation would be a good example, PHP less so.

@luke-hill the above comment also applies to you.

Thanks! The small suite of tests will be copied here.
Java tests seem must not be directly ported because the model is generated from schemas directly. So many tests will just test the generator itself(it has a much wider suite of tests)

Indeed. Most of the tests in Java are for serialization rather than the shape of the messages.

For example enums must be serialized by name, null fields must be omitted, optionals types are elided, ect. This will depend a bit on what Python offers out of the box.

It's worth pointing out here @elchupanebrej the way in which Rien is describing things is that we should / can use the CCK to test the generation. But we shouldn't have the generation "requiring on" the CCK. Hope that makes sense / apologies if I'm repeating something already understood.

i.e. for ruby here - https://github.com/cucumber/messages/blob/main/ruby/cucumber-messages.gemspec we have no direct dependencies, but we use the CCK as a development dependency (I.e. to test the generation has worked).

Apologies if this doesn't make sense.

PyPi releases don't allow dependencies on github repositories, so I can't add resources directly. If you go through commits you will see an example of tests with direct downloading CCK data. If you have better ideas how to integrate - please share your thoughts

Surely this isn't a problem if it's listed as a dev dependency as it won't be in the pypi package? (I could be wrong!)

@luke-hill, this implementation isn't dependent on CCK for generation, it was dependent on CCK for test purposes only

mpkorstanje · 2024-09-04T21:26:39Z

Left a few quick remarks, will have to take a deeper look later.

python/Makefile

luke-hill · 2024-09-05T13:22:06Z

python/src/message_samples/minimal/minimal.feature.ndjson

@@ -0,0 +1,12 @@
+{"meta":{"ci":{"buildNumber":"154666429","git":{"remote":"https://github.com/cucumber-ltd/shouty.rb.git","revision":"99684bcacf01d95875834d87903dcb072306c9ad"},"name":"GitHub Actions","url":"https://github.com/cucumber-ltd/shouty.rb/actions/runs/154666429"},"cpu":{"name":"x64"},"implementation":{"name":"fake-cucumber","version":"16.3.0"},"os":{"name":"darwin","version":"22.4.0"},"protocolVersion":"22.0.0","runtime":{"name":"node.js","version":"19.7.0"}}}


I think this is a good process what you've done here. Just commenting for documentation.

I think as/when you have gotten this all working, it would be good to migrate this and others to the CCK proper. WDYT? (Maybe something for 2025?)

migrate this and others to the CCK proper
It must work with CCK now in all possible cases. If it doesn't - let write tests & fix

@elchupanebrej As a test I'm not happy with a "sample test". As said before this create a circular dependency between the code that generates the samples and messages.

Tests for messages can be limited to testing whether the code was generated and serialization works correctly. This is does not test those things specifically while still testing many other - less relevant things.

@luke-hill what exactly do you mean by "migrating this and others to the cck"?

@mpkorstanje sorry for bothering you, it seems I can't catch a point:

Samples of messages in the CCK repository are stored as examples. Every tool that uses messages has to use them (at least serialize when some event is emitted, and deserialize when this message comes to some reporter). So I took the full suite of test data from the CCK repo and checked that the models generated were successfully parsed that messages into the model, and after that deserialized them to totally the same JSON. Could you please describe more precisely what kind of tests would be OK: would be enough if some model(for every kind of message) would be created, serialized and deserialized perfectly to the totally same model?

The CCK uses the messages to generate the output of a canonical cucumber execution. For this is needs the messages. The value the CCK adds isn't that it generates a sample of each messages, but rather that the collection of messages as a whole. So it can for example express relationships between messages.

This dependency also means that it can't be used as test data in messages. That would result in a circular dependency.

Now for messages the exact testing strategy depends on the framework and language used.

For example for Javascript, the object and it's json representation are almost identical so there is little to test at all. And because the code is generated, it doesn't seem nesesary to test every message either.

So you can see we do a round trip test of one moderately complex message and not much more.

https://github.com/cucumber/messages/blob/main/javascript/test/messagesTest.ts

For Java serialization is more complicated. It does not have a concept of undefined. So we got tests to check for that.

https://github.com/cucumber/messages/blob/main/java/src/test/java/io/cucumber/messages/NdjsonSerializationTest.java

Now I don't know enough about Python to tell you exactly what to test. I can't tell you about pitfalls I don't know about. But I imagine if third party code generator is used, a simple round trip should be enough.

mpkorstanje

There seems to have been a misunderstanding.

So just to clarify.

Either:

Source is generated by the Ruby codegen script
Generated source is checked in

Or:

Source is generated by the python build process.
Generated source is not checked in.
Make targets print a message that code code gen is handled by Python.

Which option are you going for now?

.github/workflows/test-python.yml

python/Makefile

python/pyproject.toml

mpkorstanje · 2024-09-08T11:39:41Z

python/src/message_samples/minimal/minimal.feature.ndjson

@@ -0,0 +1,12 @@
+{"meta":{"ci":{"buildNumber":"154666429","git":{"remote":"https://github.com/cucumber-ltd/shouty.rb.git","revision":"99684bcacf01d95875834d87903dcb072306c9ad"},"name":"GitHub Actions","url":"https://github.com/cucumber-ltd/shouty.rb/actions/runs/154666429"},"cpu":{"name":"x64"},"implementation":{"name":"fake-cucumber","version":"16.3.0"},"os":{"name":"darwin","version":"22.4.0"},"protocolVersion":"22.0.0","runtime":{"name":"node.js","version":"19.7.0"}}}


@elchupanebrej As a test I'm not happy with a "sample test". As said before this create a circular dependency between the code that generates the samples and messages.

Tests for messages can be limited to testing whether the code was generated and serialization works correctly. This is does not test those things specifically while still testing many other - less relevant things.

@luke-hill what exactly do you mean by "migrating this and others to the cck"?

mpkorstanje · 2024-09-08T11:40:33Z

python/src/message_samples/minimal/minimal.feature.ts

+import { Given } from '@cucumber/fake-cucumber'
+
+Given('I have {int} cukes in my belly', function (cukeCount: number) {
+  assert(cukeCount)


This file seems unused in any tests.

mpkorstanje · 2024-09-08T11:40:54Z

python/src/messages.py

@@ -0,0 +1,3 @@
+from _messages import *
+
+ExpressionType = Type1


I don't understand what this file does. Can you explain?

We have two entities in the original model named Type (design bug from my perspective). This module is a simple adapter, so the end user will import Type and ExpressionType but not Type and Type1. In the serialized model they both are named Type as it was in the original model

Would it be possible to fix that in the code generator instead?

And if it is not possible, an explanatory comment would be useful.

python/src/_messages.py

python/src/message_samples/__init__.py

youtux · 2024-09-15T16:01:09Z

python/pyproject.toml

+]
+dependencies = [
+  "importlib_resources",
+  "pydantic>=2.0.3"


Is it really necessary to use and add pydantic as a dependency ?
Many people are still on pydantic v1, and this would require pytest-bdd users to upgrade to pydantic v2 since pytest-bdd will soon depend on gherkin

Aren’t stdlib dataclasses enough?

importlib_resources is also, from what I can see, only used for tests which I'm not sure is needed either

@youtux Yes, this is technically possible, but such realization will be dependent on some library like https://github.com/lidatong/dataclasses-json (the best option for now), which are not as good supported as pydantic

From another perspective - testing utilities are selected at the start of a project, so if the messages package will be used somewhere - it most probably would be dependent on the new version of Pydantic

but there are many projects using pytest-bdd for years, and this would be an issue.
We can do without pydantic in a very simple way. We can use data classes, then when we need to serialise to json we call asdict(model). If we need custom encoders (e.g. for date times) we can implement a simple JsONEncoder and pass that to json.dumps(asdict(model), encoder=…).

Or also just implement custom serialiser for each object

in this case, we have to implement dict_factory for dataclass.asdict, which will have to take in count Enums, or there would be an issue with serialization to JSON. And deserialisation to the dataclass also will be an issue (Enums again)
And pydantic covers both of this issues

Minimizing the number of dependencies avoids a potential conflict with the system under test. And it seems to me that any effort saved by using Pydantic in Cucumber will be meaningless if Cucumber can't be used because of it.

But I'm not in the Python ecosystem so I'd like to see a consensus on this problem from those who are.

I really think we should not bring in a big dependency like pydantic here, especially since it has made a big API change in v2, and I can see it make it difficult for users to adopt this library if it conflicts with their pydantic v1 requirement.

What's the use of pydantic here? I don't see it being used for serialisation / deserialisation here.
What's the API of this library going to look like?

Messages library is hardly used for serialization/deserialization, for example:

Test runner must produce messages in the ndjson format, so it uses model of "messages" lib to represent outcomes, messages lib serializes and validates against Json schema (non-directly).

Test reporter consumes ndjson stream of messages and uses "messages" library to deserialize inputs and validate them.

So "messages" lib is a bridge between test runner and test reporter (potentially from different languages ecosystems)

ok, but how is the API of this lib supposed to look like?

from cucumber_messages import ??? ???

@youtux , please check python/tests/test_model_load.py test in this PR (I'll rework tests later).

For example reporting in the pytest-bdd-ng uses this particular model:
https://github.com/elchupanebrej/pytest-bdd-ng/blob/default/src/pytest_bdd/message_plugin.py

elchupanebrej · 2024-09-17T05:41:08Z

Thanks for great review, return later this week and will update all things accordingly 😀

codecov-commenter · 2024-09-21T19:20:34Z

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

mpkorstanje

I'm sorry to see that the misunderstanding I highlighted in the last review persists. While some aspects have been addressed, they have not been addressed in full. Let me know if we need to schedule a call and talk this through.

Further more, the current test set contains many incidental details while also not testing for anything specific. This will make the tests break whenever small changes to the schema are made. Given that this repository currently hosts 9 languages, keeping tests up to date becomes tedious quickly.

Finally I'd like to see a consensus on the use of pydantic. And it might be useful to do that first as it will significantly impact the shape of this pull request.

mpkorstanje · 2024-09-22T12:30:59Z

python/Makefile

+	echo "Skipping code generation - code is generated by Python"
+
+generate-real: require install-deps
+	datamodel-codegen \


Would it be possible to move this into pythons build process?

mpkorstanje · 2024-09-22T12:31:38Z

python/Makefile

+		--target-python-version=3.8
+
+require: ## Check requirements for the code generation (python is required)
+	@python --version >/dev/null 2>&1 || (echo "ERROR: python is required."; exit 1)


Python should not be required. This can be a stub too.

mpkorstanje · 2024-09-22T12:33:21Z

python/src/cucumber_messages/messages.py

+from pydantic import BaseModel, ConfigDict, Field
+
+
+class ContentEncoding(Enum):


If the code is generated by Python, then I would not expect this file to be checked in.

mpkorstanje · 2024-09-22T12:35:15Z

.github/workflows/test-python.yml

+          - python-version: "3.10"
+            os: windows-latest
+          - python-version: "3.11"
+            os: windows-latest


There has to be a more efficient way to run run all versions on ubuntu and exclude osx and windows.

I would use an include matrix personally @elchupanebrej

mpkorstanje · 2024-09-22T12:38:12Z

python/src/messages.py

@@ -0,0 +1,3 @@
+from _messages import *
+
+ExpressionType = Type1


Would it be possible to fix that in the code generator instead?

And if it is not possible, an explanatory comment would be useful.

mpkorstanje · 2024-09-22T12:39:07Z

.github/workflows/test-python.yml

+  build:
+
+    runs-on: ${{ matrix.os }}
+    timeout-minutes: 20


This looks unnecessary.

mpkorstanje · 2024-09-22T12:49:50Z

python/tests/test_model_load.py

+
+with (resource_path / "message_samples/minimal/minimal.feature.ndjson").open(mode="r") as ast_file:
+    model_data = [*map(json.loads, ast_file)]
+oracle_models = [


This oracle is overly detailed and at the same time does not specify what property is being tested.

I reckon the important things to check are

Are null values omitted from the output

Enums are written by name

Something simple can round trip.

mpkorstanje · 2024-09-22T13:07:45Z

python/pyproject.toml

+]
+dependencies = [
+  "importlib_resources",
+  "pydantic>=2.0.3"


Minimizing the number of dependencies avoids a potential conflict with the system under test. And it seems to me that any effort saved by using Pydantic in Cucumber will be meaningless if Cucumber can't be used because of it.

But I'm not in the Python ecosystem so I'd like to see a consensus on this problem from those who are.

jsa34 · 2024-10-20T14:31:57Z

Hello @elchupanebrej !

I wondered if you had had a chance to get back to this?

😃

youtux · 2024-10-25T20:38:59Z

I'm sorry to see that the misunderstanding I highlighted in the last review persists. While some aspects have been addressed, they have not been addressed in full. Let me know if we need to schedule a call and talk this through.

Further more, the current test set contains many incidental details while also not testing for anything specific. This will make the tests break whenever small changes to the schema are made. Given that this repository currently hosts 9 languages, keeping tests up to date becomes tedious quickly.

Finally I'd like to see a consensus on the use of pydantic. And it might be useful to do that first as it will significantly impact the shape of this pull request.

To me it's still not quite clear how this implementation (but the others already present in the repo as well) is supposed to be used.
This repo should just define the json schema for the Cucumber Messages specification, but it has implementations, and it doesn't show anywhere how these implementations are supposed to be used.
I just asked the same question on Discord, maybe they will help with the answer.

youtux · 2024-10-25T21:45:47Z

If we are doing this implementations just to define the classes, I think we should better use an automated tool that converts from jsonschema to python models.

A good one seems to be https://github.com/koxudaxi/datamodel-code-generator, I managed to create dataclasses from the jsonspec like this:

 docker run --rm -v "${PWD}:/local" koxudaxi/datamodel-code-generator --input /local/jsonschema/GherkinDocument.json  --output /local/model.py --output-model-type dataclasses.dataclass

Generated `model.py` using dataclass generator

# generated by datamodel-codegen:
#   filename:  GherkinDocument.json
#   timestamp: 2024-10-25T21:41:54+00:00

from __future__ import annotations

from dataclasses import dataclass
from enum import Enum
from typing import List, Optional


@dataclass
class Location:
    line: int
    column: Optional[int] = None


@dataclass
class Comment:
    location: Location
    text: str


@dataclass
class DocString:
    location: Location
    content: str
    delimiter: str
    mediaType: Optional[str] = None


class KeywordType(Enum):
    Unknown = 'Unknown'
    Context = 'Context'
    Action = 'Action'
    Outcome = 'Outcome'
    Conjunction = 'Conjunction'


@dataclass
class TableCell:
    location: Location
    value: str


@dataclass
class TableRow:
    location: Location
    cells: List[TableCell]
    id: str


@dataclass
class Tag:
    location: Location
    name: str
    id: str


@dataclass
class DataTable:
    location: Location
    rows: List[TableRow]


@dataclass
class Examples:
    location: Location
    tags: List[Tag]
    keyword: str
    name: str
    description: str
    tableBody: List[TableRow]
    id: str
    tableHeader: Optional[TableRow] = None


@dataclass
class Step:
    location: Location
    keyword: str
    text: str
    id: str
    keywordType: Optional[KeywordType] = None
    docString: Optional[DocString] = None
    dataTable: Optional[DataTable] = None


@dataclass
class Background:
    location: Location
    keyword: str
    name: str
    description: str
    steps: List[Step]
    id: str


@dataclass
class Scenario:
    location: Location
    tags: List[Tag]
    keyword: str
    name: str
    description: str
    steps: List[Step]
    examples: List[Examples]
    id: str


@dataclass
class RuleChild:
    background: Optional[Background] = None
    scenario: Optional[Scenario] = None


@dataclass
class Rule:
    location: Location
    tags: List[Tag]
    keyword: str
    name: str
    description: str
    children: List[RuleChild]
    id: str


@dataclass
class FeatureChild:
    rule: Optional[Rule] = None
    background: Optional[Background] = None
    scenario: Optional[Scenario] = None


@dataclass
class Feature:
    location: Location
    tags: List[Tag]
    language: str
    keyword: str
    name: str
    description: str
    children: List[FeatureChild]


@dataclass
class Model:
    comments: List[Comment]
    uri: Optional[str] = None
    feature: Optional[Feature] = None

Generated `model.py` using pydantic v2 generator

# generated by datamodel-codegen:
#   filename:  GherkinDocument.json
#   timestamp: 2024-10-25T21:49:23+00:00

from __future__ import annotations

from enum import Enum
from typing import List, Optional

from pydantic import BaseModel, ConfigDict, Field


class Location(BaseModel):
    model_config = ConfigDict(
        extra='forbid',
    )
    line: int
    column: Optional[int] = None


class Comment(BaseModel):
    model_config = ConfigDict(
        extra='forbid',
    )
    location: Location = Field(..., description='The location of the comment')
    text: str = Field(..., description='The text of the comment')


class DocString(BaseModel):
    model_config = ConfigDict(
        extra='forbid',
    )
    location: Location
    mediaType: Optional[str] = None
    content: str
    delimiter: str


class KeywordType(Enum):
    Unknown = 'Unknown'
    Context = 'Context'
    Action = 'Action'
    Outcome = 'Outcome'
    Conjunction = 'Conjunction'


class TableCell(BaseModel):
    model_config = ConfigDict(
        extra='forbid',
    )
    location: Location = Field(..., description='The location of the cell')
    value: str = Field(..., description='The value of the cell')


class TableRow(BaseModel):
    model_config = ConfigDict(
        extra='forbid',
    )
    location: Location = Field(
        ..., description='The location of the first cell in the row'
    )
    cells: List[TableCell] = Field(..., description='Cells in the row')
    id: str


class Tag(BaseModel):
    model_config = ConfigDict(
        extra='forbid',
    )
    location: Location = Field(..., description='Location of the tag')
    name: str = Field(
        ..., description='The name of the tag (including the leading `@`)'
    )
    id: str = Field(
        ..., description='Unique ID to be able to reference the Tag from PickleTag'
    )


class DataTable(BaseModel):
    model_config = ConfigDict(
        extra='forbid',
    )
    location: Location
    rows: List[TableRow]


class Examples(BaseModel):
    model_config = ConfigDict(
        extra='forbid',
    )
    location: Location = Field(
        ..., description='The location of the `Examples` keyword'
    )
    tags: List[Tag]
    keyword: str
    name: str
    description: str
    tableHeader: Optional[TableRow] = None
    tableBody: List[TableRow]
    id: str


class Step(BaseModel):
    model_config = ConfigDict(
        extra='forbid',
    )
    location: Location = Field(..., description="The location of the steps' `keyword`")
    keyword: str = Field(
        ..., description='The actual keyword as it appeared in the source.'
    )
    keywordType: Optional[KeywordType] = Field(
        None,
        description="The test phase signalled by the keyword: Context definition (Given), Action performance (When), Outcome assertion (Then). Other keywords signal Continuation (And and But) from a prior keyword. Please note that all translations which a dialect maps to multiple keywords (`*` is in this category for all dialects), map to 'Unknown'.",
    )
    text: str
    docString: Optional[DocString] = None
    dataTable: Optional[DataTable] = None
    id: str = Field(
        ..., description='Unique ID to be able to reference the Step from PickleStep'
    )


class Background(BaseModel):
    model_config = ConfigDict(
        extra='forbid',
    )
    location: Location = Field(
        ..., description='The location of the `Background` keyword'
    )
    keyword: str
    name: str
    description: str
    steps: List[Step]
    id: str


class Scenario(BaseModel):
    model_config = ConfigDict(
        extra='forbid',
    )
    location: Location = Field(
        ..., description='The location of the `Scenario` keyword'
    )
    tags: List[Tag]
    keyword: str
    name: str
    description: str
    steps: List[Step]
    examples: List[Examples]
    id: str


class RuleChild(BaseModel):
    model_config = ConfigDict(
        extra='forbid',
    )
    background: Optional[Background] = None
    scenario: Optional[Scenario] = None


class Rule(BaseModel):
    model_config = ConfigDict(
        extra='forbid',
    )
    location: Location = Field(..., description='The location of the `Rule` keyword')
    tags: List[Tag] = Field(
        ..., description='All the tags placed above the `Rule` keyword'
    )
    keyword: str
    name: str
    description: str
    children: List[RuleChild]
    id: str


class FeatureChild(BaseModel):
    model_config = ConfigDict(
        extra='forbid',
    )
    rule: Optional[Rule] = None
    background: Optional[Background] = None
    scenario: Optional[Scenario] = None


class Feature(BaseModel):
    model_config = ConfigDict(
        extra='forbid',
    )
    location: Location = Field(..., description='The location of the `Feature` keyword')
    tags: List[Tag] = Field(
        ..., description='All the tags placed above the `Feature` keyword'
    )
    language: str = Field(
        ...,
        description='The [ISO 639-1](https://en.wikipedia.org/wiki/ISO_639-1) language code of the Gherkin document',
    )
    keyword: str = Field(
        ...,
        description='The text of the `Feature` keyword (in the language specified by `language`)',
    )
    name: str = Field(
        ..., description='The name of the feature (the text following the `keyword`)'
    )
    description: str = Field(
        ...,
        description='The line(s) underneath the line with the `keyword` that are used as description',
    )
    children: List[FeatureChild] = Field(..., description='Zero or more children')


class Model(BaseModel):
    model_config = ConfigDict(
        extra='forbid',
    )
    uri: Optional[str] = Field(
        None,
        description='*\n The [URI](https://en.wikipedia.org/wiki/Uniform_Resource_Identifier)\n of the source, typically a file path relative to the root directory',
    )
    feature: Optional[Feature] = None
    comments: List[Comment] = Field(
        ..., description='All the comments in the Gherkin document'
    )

It also supports pydantic v1/v2, and other libs, if one needs that.
A this point, I guess we should just create a README.md that explains users that want to have python classes from the JSONSchema how to do that with this tool.

youtux · 2024-10-25T22:03:43Z

If anything, we could maintain the generated files into this repo, and make sure every time the json schema is updated these files are regenerated, so that other downstream users can use whatever flavour of models they want.

elchupanebrej · 2024-10-28T17:31:20Z

we should better use an automated tool that converts from jsonschema to python models.

@youtux, this approach and tool are used here exactly! Check Makefile pls!

mpkorstanje · 2024-10-28T21:17:54Z

To me it's still not quite clear how this implementation (but the others already present in the repo as well) is supposed to be used.

A schema definition isn't any good without data objects to go with it and their use is mostly to provide a type safe representation of the message.

For example

https://github.com/cucumber/junit-xml-formatter/blob/main/java/src/main/java/io/cucumber/junitxmlformatter/XmlReportWriter.java

And while in theory each library could generate dtos based of the schema, that isn't practical once libraries start calling each other. So having a shared implementation of the data objects is essential.

youtux · 2024-10-28T21:33:01Z

Got it. Then I'd propose for the python impl to provide at least the data classes versions, since its the most compatible one, and possibly also the pydantic version under a different module, so that downstream users can choose what to use

jsa34 · 2024-11-07T17:51:00Z

Is there anything I can help with, as I'd love to try and messages over the line to support our work with gherkin?

luke-hill

I'm going to step away from this now I think as there is enough voices commenting on it. I don't quite understand why we've not gone down the route that the other 8 languages have done with using the codegen tool given I spent a while refactoring it so now it's a tiny class you need to make 🤷

luke-hill · 2024-11-08T09:10:46Z

.github/workflows/test-python.yml

+          - python-version: "3.10"
+            os: windows-latest
+          - python-version: "3.11"
+            os: windows-latest


I would use an include matrix personally @elchupanebrej

elchupanebrej force-pushed the python-impl branch from 18b7d09 to 6a26520 Compare July 18, 2023 18:10

mpkorstanje requested changes Jul 18, 2023

View reviewed changes

elchupanebrej marked this pull request as draft July 19, 2023 15:08

luke-hill requested changes Nov 1, 2023

View reviewed changes

python/pyproject.toml Show resolved Hide resolved

.github/workflows/test-python.yml Show resolved Hide resolved

elchupanebrej force-pushed the python-impl branch 2 times, most recently from ee63f2a to 358b36b Compare December 31, 2023 18:50

luke-hill reviewed Jan 4, 2024

View reviewed changes

python/RELEASING.md Outdated Show resolved Hide resolved

elchupanebrej mentioned this pull request Jan 12, 2024

Add python implemeatation to official messages library elchupanebrej/pytest-bdd-ng#104

Open

elchupanebrej force-pushed the python-impl branch 2 times, most recently from 3256104 to effdd2b Compare September 4, 2024 20:54

mpkorstanje reviewed Sep 4, 2024

View reviewed changes

mpkorstanje marked this pull request as ready for review September 4, 2024 21:26

mpkorstanje reviewed Sep 4, 2024

View reviewed changes

python/Makefile Show resolved Hide resolved

luke-hill reviewed Sep 5, 2024

View reviewed changes

elchupanebrej force-pushed the python-impl branch 2 times, most recently from ae519d7 to 99e72d6 Compare September 7, 2024 15:39

mpkorstanje requested changes Sep 8, 2024

View reviewed changes

mpkorstanje reviewed Sep 8, 2024

View reviewed changes

python/src/_messages.py Outdated Show resolved Hide resolved

jsa34 reviewed Sep 15, 2024

View reviewed changes

python/src/message_samples/__init__.py Outdated Show resolved Hide resolved

youtux reviewed Sep 15, 2024

View reviewed changes

elchupanebrej force-pushed the python-impl branch from c127cc1 to f6ecf72 Compare September 21, 2024 10:55

elchupanebrej force-pushed the python-impl branch 9 times, most recently from 5d79e6f to 11a8b21 Compare September 21, 2024 18:04

elchupanebrej added 4 commits September 21, 2024 21:08

Add messages implementation for python

88ec5b6

Use CCK examples load on test

3389a62

Bare minimum tests for python model

3da9e8f

Move out code generation from general flow

af36ce3

elchupanebrej force-pushed the python-impl branch from 11a8b21 to 5052824 Compare September 21, 2024 18:08

mpkorstanje self-requested a review September 21, 2024 18:20

Rework of test for minimal Feature

614d55c

elchupanebrej force-pushed the python-impl branch from 5052824 to 614d55c Compare September 21, 2024 19:16

mpkorstanje requested changes Sep 22, 2024

View reviewed changes

jsa34 mentioned this pull request Oct 25, 2024

Feature/cucumber junit report pytest-dev/pytest-bdd#729

Open

mpkorstanje mentioned this pull request Nov 5, 2024

Python cuke-messages bad Pydantic model for ExceptionMessage #263

Closed

luke-hill reviewed Nov 8, 2024

View reviewed changes

		@@ -0,0 +1,12 @@
		{"meta":{"ci":{"buildNumber":"154666429","git":{"remote":"https://github.com/cucumber-ltd/shouty.rb.git","revision":"99684bcacf01d95875834d87903dcb072306c9ad"},"name":"GitHub Actions","url":"https://github.com/cucumber-ltd/shouty.rb/actions/runs/154666429"},"cpu":{"name":"x64"},"implementation":{"name":"fake-cucumber","version":"16.3.0"},"os":{"name":"darwin","version":"22.4.0"},"protocolVersion":"22.0.0","runtime":{"name":"node.js","version":"19.7.0"}}}

		@@ -0,0 +1,3 @@
		from _messages import *

		ExpressionType = Type1

		from pydantic import BaseModel, ConfigDict, Field


		class ContentEncoding(Enum):

Add messages implementation for python #165

Are you sure you want to change the base?

Add messages implementation for python #165

Conversation

elchupanebrej commented Jul 18, 2023

🤔 What's changed?

🏷️ What kind of change is this?

📋 Checklist:

elchupanebrej commented Jul 18, 2023

mpkorstanje left a comment • edited Loading

Choose a reason for hiding this comment

elchupanebrej commented Jul 19, 2023

mpkorstanje commented Jul 19, 2023 • edited Loading

luke-hill left a comment

Choose a reason for hiding this comment

mpkorstanje commented Jan 4, 2024 • edited Loading

luke-hill commented Apr 4, 2024

elchupanebrej commented Sep 4, 2024

elchupanebrej commented Sep 4, 2024

mpkorstanje Sep 4, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mpkorstanje Sep 4, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jsa34 Sep 16, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mpkorstanje commented Sep 4, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mpkorstanje Oct 28, 2024 • edited Loading

Choose a reason for hiding this comment

mpkorstanje left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elchupanebrej commented Sep 17, 2024

codecov-commenter commented Sep 21, 2024

Welcome to Codecov 🎉

mpkorstanje left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jsa34 commented Oct 20, 2024 • edited Loading

youtux commented Oct 25, 2024

youtux commented Oct 25, 2024 • edited Loading

youtux commented Oct 25, 2024

elchupanebrej commented Oct 28, 2024

mpkorstanje commented Oct 28, 2024 • edited Loading

youtux commented Oct 28, 2024

jsa34 commented Nov 7, 2024 • edited Loading

luke-hill left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mpkorstanje left a comment •

edited

Loading

mpkorstanje commented Jul 19, 2023 •

edited

Loading

mpkorstanje commented Jan 4, 2024 •

edited

Loading

mpkorstanje Sep 4, 2024 •

edited

Loading

mpkorstanje Sep 4, 2024 •

edited

Loading

jsa34 Sep 16, 2024 •

edited

Loading

mpkorstanje Oct 28, 2024 •

edited

Loading

mpkorstanje left a comment •

edited

Loading

jsa34 commented Oct 20, 2024 •

edited

Loading

youtux commented Oct 25, 2024 •

edited

Loading

mpkorstanje commented Oct 28, 2024 •

edited

Loading

jsa34 commented Nov 7, 2024 •

edited

Loading