Cucumber Expressions for Python

[jsa34]

Description

Cucumber expressions for Python

Motivation & context

Fixes #27

Type of change

• New feature (non-breaking change which adds new behaviour)

Checklist:

• I have read the CONTRIBUTING document.
• My change needed additional tests
  • I have added tests to cover my changes.
• My change requires a change to the documentation.
  • I have updated the documentation accordingly.
• I have added an entry to the "Unreleased" section of the CHANGELOG, linking to this pull request.

[jsa34]

I may have missed some bits, but wanted to get the first "draft" out for feedback.

I can't work out the only test that fails: test_prefers_expression_with_longest_non_empty_match

[jenisys]

I just look over your pull-request (not the contents, just the filename).

You violated the PEP-8 naming conventions for Python by using CamelCase instead of snake_case for Python packages/modules.

• python/Cucumber/CucumberExpressions/ => SHOULD BE: python/cucumber_expressions/

  REASONS:

  • Overly and unnecessarily complex
  • Naming style: You have a "tautology" (you name "cucumber" twice during imports; "Doppelt gemoppelt)
  • UNVERIFIED: Your "Cucumber" is probably not a Python namespace module. Therefore, it will block others from using the "Cucumber" prefix/package-name.

SEE ALSO:

• PEP-0008: Style Guide for Python Code

OTHERWISE:

• Unit tests within the package: I would normally move them out of the package

  REASONS:

  • The installed package is smaller (less "FAT").
  • The installed tests within the installed package are nearly never used.

[jsa34]

Great feedback, @jenisys - thanks!

I am not an expert in Python so thought I would pop what I have done so far rather than getting too far down the line and not realising my mistakes.

I will have a look at what you have suggested - having never released any artifacts for python, all that area and nomenclature are new territory.

Many thanks again!

[jenisys]

No problem, learning by doing.
Keep up your good work 😀

[blaisep]

Outstanding effort @jsa34 , thanks so much for contributing. I rounded up some pythonistas to help with this effort just before the holidays and I think everyone was too busy to take action.
@mattwynne suggested that we try to make the python port adhere to the "pythonic" conventions and it's wonderful that @jenisys provides guidance.

I would be very happy to work with you if you are interested in any kind of collaboration.

Great feedback, @jenisys - thanks!

I am not an expert in Python so thought I would pop what I have done so far rather than getting too far down the line and not realising my mistakes.

I will have a look at what you have suggested - having never released any artifacts for python, all that area and nomenclature are new territory.

Many thanks again!

[jsa34]

Outstanding effort @jsa34 , thanks so much for contributing. I rounded up some pythonistas to help with this effort just before the holidays and I think everyone was too busy to take action. @mattwynne suggested that we try to make the python port adhere to the "pythonic" conventions and it's wonderful that @jenisys provides guidance.

I would be very happy to work with you if you are interested in any kind of collaboration.

Great feedback, @jenisys - thanks!
I am not an expert in Python so thought I would pop what I have done so far rather than getting too far down the line and not realising my mistakes.
I will have a look at what you have suggested - having never released any artifacts for python, all that area and nomenclature are new territory.
Many thanks again!

Hello!

That's great and would be very much appreciated.

I tried to keep as close to the other lang implementations as possible with a sprinkling of pythonisms where possible, but where the line is drawn I'm not sure as the closer it is to Ruby/JS, etc I guess maintainability for cross language feature is better, but again I deliberately thought I would share at this point so I can start discussions and gather feedback.

I would very much appreciate any support/collaboration as it will most definitely help me learn more about Python and hopefully help me become more "pythonic"!

Let me know what you suggest :)

[blaisep]

@jsa34 , I would say let's go ahead and merge main into your PR so that you can resolve the merge conflicts (a courtesy to @jenisys ).

I think that @mattwynne is correct is wanting to make the language-specific implementations representative of the native customs.

[jsa34]

@blaisep Resolved merge conflicts.

Happy to make this reflect more the language native customs, but if you are able to provide any guidance/examples of this that would be gratefully appreciated!

Happy to chat about it or to go through code comments

[aslakhellesoy]

Excellent work @jsa34 - thank you!

Could you please add build scripts and/or instructions so other contributors can run all tests with a single command after cloning the repo?

• Maybe add a requirements.txt file that lists dependencies
• How to run all the tests

Could you also please add a GitHub Action that does all of the above? You can look at this one for inspiration: https://github.com/cucumber/tag-expressions/blob/main/.github/workflows/test-python.yml

@jenisys I'm not sure it's a good idea to rely on make for this (Windows users don't always have it). What's the most idiomatic and cross platform tool in Python to write build scripts?

[jenisys]

@aslakhellesoy (question from review above):

@jenisys I'm not sure it's a good idea to rely on make for this (Windows users don't always have it). What's the most
idiomatic and cross platform tool in Python to write build scripts?

Correct, using "make" normally does not work on Windows for many users.
I normally used invoke (https://github.com/pyinvoke/invoke), as a small build tool.

Otherwise, if a "pyproject.toml" is provide (newer mechanism instead of "setup.py") to be able to install the package,
you can use as build script for developer tasks (with: poetry or ...; I haven't used it much yet).

BUT: Just for testing it, a simpler solution would be a short description in the README.

$ cd cucumber-expressions/python/

# -- BETTER: Use "requirements.txt" (or: "py.requirements.txt")
# HINT: Newer pip versions install under $HOME
# MAYBE NEEDED: PATH="$HOME/Library/Python/3.9/bin:$PATH" (for installed python scripts on my Mac)
$ pip install pytest coverage PyYAML

# -- Run the tests:
$ pytest

# -- Run test with code coverage:
$ coverage run -m pytest

# -- AFTERWARDS:
$ coverage report
...

REVIEW NOTE @jsa34:

• I am currently missing a mechanism to install the package, like: "setup.py" file or "pyproject.toml" file !?!
• I will look into the other question above (if the mechanism is pythonic).

[jenisys]

NOTE: The cucumber-expressions/python/test/README.md contains a description how to run the tests (like my short description above). Maybe this description should be moved up one level into the cucumber-expressions/python/README.md.

[aslakhellesoy]

I think this is good to go now. Any objections if we merge it?

We still need a GA workflow for releasing the Python code, but I think that should be done separately from this PR.

[jenisys]

@aslakhellesoy @jsa34
There are two options:

• Merge it now and continue or
• Provide some fixes and merge then

NOTES:

• I ran "pylint" over the code yesterday and it stumbled over a circular dependency problems and some other issues (besides the noise that docstring/documentation is missing)
• Currently, a method to install the package in Python is missing: No "setup.py" or "pyproject.toml"
• I try to look over the remaining stuff tonight

[aslakhellesoy]

Great work!! Just waiting for @jenisys' final changes/feedback now.

[blaisep]

@jsa34 , Sorry I have been AFK

Let me know what you suggest :)

This is really very nice work!!

OK, it took me an hour to find something I could contribute...

Pathlib: because paths are not strings

Since @aslakhellesoy mentioned Windows users, consider using pathlib instead of os.path. Trey Hunner has a well-known post about the problems with os module.

Python can now join the README

Consider extending the list of examples in the README to include Python.

[jsa34]

I have responded to @blaisep 's feedback. I already had updated the README.md entry (I hope!) - thanks, @blaisep !

@jenisys - good points! Updated for pylint comment: I sacrificed a type hint to avoid circular dependency hell! I had moved the import to be inline to avoid this, but better to be safe than sorry!

I hadn't considered the build part in this scope, and have never had to release anything, but just learned about Poetry and seemed like a great idea to use (learning so much doing this!) I have updated this to use poetry and so if this accepted I believe adding the deployment script should be trivial, but never released anything on Pypi so thought I would hold fire going that far :)

[jenisys]

@aslakhellesoy @jsa34
Ok, I had a short look over the current state (not a complete review).
Most of the problems/smells from the "pylint" run seem to by solved by the latest changes from @jsa34 .

NECESSARY BEFORE THE MERGE (in my opinion):

• "pyproject.toml" exists now
• P1
• P2
• OPTIONAL: P3 should be discussed or fixed

DETAILS:

• P1. Running pytest with Python 3.9 on my Mac, I get a DeprecationWarning on:
  cucumber_expressions/cucumber_expression.py:19
ESCAPE_PATTERN = b"([\\^\[({$.|?*+})\]])" # noqa W605
  SHOULD BE:
  ESCAPE_PATTERN = rb"([\\^\[({$.|?*+})\]])" # noqa W605

  HINT:

  • That fixes also the "pylint" warning(s)
  • Regular expression checks (what is complained about and what not) has changed in the last few Python versions.
    I think it started with Python 3.9, and continued with Python 3.10.

• P2. BAD_DOCSTRING: docstring is in the wrong position:
  Class docstring position is used for the description of the Class ctor (__init__())
  EXAMPLES:

  • cucumber_expressions.parameter_type:ParameterType
  • cucumber_expressions.regular_expression:RegularExpression
  • ...

• P3. NAMING: TAUTOLOGY (naming is unnecessarily complex; same name is repeated on different levels):

  • RENAME: "cucumber_expressions/cucumber_expression.py" to "cucumber_expressions/expression.py"
  • RENAME: "cucumber_expressions/cucumber_expression_generator.py" to "cucumber_expressions/expression_generator.py"
  • RENAME: "cucumber_expressions/cucumber_expression_parser.py" to "cucumber_expressions/expression_parser.py"
  • RENAME: "cucumber_expressions/cucumber_expression_tokenizer.py" to "cucumber_expressions/expression_tokenizer.py"

• P4. BAD-NAME: Argument name "at" -- BAD NAME (in my opinion), BETTER: pos (or: position)
  -- Meant is an index/position within a list.

• P5. MINOR: Module structure is not really Pythonic.
  SMELL: Each module contains just one class with the same name as module, just in CamelCase.
  EXPLANATION: In Python, the module structure is normally not so fine grained.
  In addition, you have multiple modules that seem to form a group (have same prefix).
  Therefore, these seem to belong together (on first sight).

• P6. MINOR: Constant regexp patterns are not compiled
  HINT: Constant regular expression patterns are faster to use if you compile them.
  AFFECTED: cucumber_expressions.parameter_type:ParameterType, ...
  NAMING: Same "regexp" is used for different things:
  The regexp-pattern (as string) and the regexp-object (as fast matcher)

• P7. COULD BE IMPROVED: Documentation of source code (docstring): Nearly non-existing.
  => Should be improved in the future, especially for module- and class-descriptions.

[blaisep]

@jsa34 ,

but never released anything on Pypi so thought I would hold fire going that far :)
Many pythonistas use flit to package modules for pypi because it's almost plug n play ... so you might want to consider that to get started.

I agree with Jens about the p3 naming because it is nicer to say:
from cucumber-expressions import expressions
than
from cucumber_expressions import cucumber_expressions

(I think there is also a namespace restriction of hyphens in module names)

[jsa34]

Hello!

Thank you once again for great feedback.

I have address what I can, but I agree the documentation needs to be looked properly, which would be good to come back to with a follow up MR as I don't have capacity at the moment.

@jenisys : I have addressed points 1-3 (thank you!)

I have added a release yml (provisionally), but it obviously needs secrets set up.

Any other feedback, as always, much appreciated!

[blaisep]

Hi @jsa34 , my schedule is more open this week, and I would be very happy to help with the documentation as well as to borrow some of the cleverness @jenisys does with invoke

It may be easier to do this realtime. You can reach me at home: blaise at gmail dot com
or drop time on my calendar.
I do a lot of pair and mob programming, for example

I have address what I can, but I agree the documentation needs to be looked properly, which would be good to come back to with a follow up MR as I don't have capacity at the moment.

[aslakhellesoy]

Is anything holding this PR up? If there are remaining issues to be fixed, can we do that in a new PR?

[jenisys]

AFAIK: no. Any remaining things can be done afterwards (if necessary).