Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add typing hints #1125

Open
rbuffat opened this issue Jul 8, 2022 · 9 comments
Open

Add typing hints #1125

rbuffat opened this issue Jul 8, 2022 · 9 comments

Comments

@rbuffat
Copy link
Contributor

rbuffat commented Jul 8, 2022

Recently it seems that more and more Python libraries include typing hints. Personally, I'm a fan of type hints while developing. Are there any plans to add typing hints to Fiona?

I'm not 100% sure what the best way to deal with the dynamic nature of the dataset schemas is. Maybe the implementation of static typing of psycopg3 could be used as inspiration: https://www.psycopg.org/psycopg3/docs/advanced/typing.html

@sgillies
Copy link
Member

There wasn't much point in doing this before the Feature, Geometry, Properties classes. They will help quite a bit. Maybe a Schema class would, too?

@StefanBrand
Copy link
Contributor

In a naive attempt to model the parameters to fiona.open (v1.8) we created this:

from typing import TypedDict

class FionaBaseDict(TypedDict):
    properties: dict[str, Any]


class FionaSchemaWithoutGeometry(FionaBaseDict):
    geometry: None


class FionaSchemaWithGeometry(FionaBaseDict):
    geometry: str


FionaSchema = Union[FionaSchemaWithoutGeometry, FionaSchemaWithGeometry]


class RecordGeometry(TypedDict):
    type: str
    coordinates: tuple


class RecordWithoutGeometry(FionaBaseDict):
    geometry: None


class RecordWithGeometry(FionaBaseDict):
    geometry: RecordGeometry


Record = Union[RecordWithoutGeometry, RecordWithGeometry]

I'm not yet familiar with the new Feature, Geometry, Properties classes. That would probably simplify things. In any case I would be interested to help get type hints into fiona, either by sharing opinion or contributing code.

@sgillies
Copy link
Member

sgillies commented Feb 9, 2023

@StefanBrand @rbuffat what would you think about collaborating on stubs for 1.9 and then possibly moving the stubs into the code at 2.0?

@rbuffat
Copy link
Contributor Author

rbuffat commented Feb 12, 2023

@sgillies I can have a go at it but probably during the next few weeks I don't have too much time available, thus it can take a while. I will also need to dive a bit deeper into the Python typing system and maybe look at how other projects are implementing dynamic types (Sqlalchemy or pydantic come to my mind here, perhaps it would also be worth asking somebody familiar with mypy/pyright for some inputs). Being able to static type check writerecords() or the collection iterator would probably be the most useful thing, but I guess also the hardest to implement.

@StefanBrand
Copy link
Contributor

but probably during the next few weeks I don't have too much time available

Maybe that's not too bad in this case. I myself should have more availability starting from (mid?) April. I haven't much worked with type stubs instead putting type hints directly into .py scripts. It will be an interesting exercise. If time permits I will look into type stub implementation.

@StefanBrand
Copy link
Contributor

StefanBrand commented Mar 8, 2023

Interesting opinion piece by @Tinche, arguing that "Python is two languages now": untyped Python (used for infrastructure code) and typed Python (used in business logic). In order to combine the two, he argues that (bold by me)

we need infrastructure code not to be type-hinted internally, but to provide type-hinted interfaces at the code boundaries. This is exactly where the ecosystem is going, with noteworthy examples being SQLAlchemy 2.0 and a new generation of web frameworks like FastAPI.

@Tinche's argument is in line with @rbuffat's comment:

Being able to static type check writerecords() or the collection iterator would probably be the most useful thing, but I guess also the hardest to implement.


On another note, @constantinius and me have submitted a lightning talk proposal to FOSS4G 2023: Adding static type hints to fiona :: FOSS4G 2023 :: pretalx (currently in the community voting process). If it is accepted and you, @rbuffat, and any other contributors will attend FOSS4G, I will also be happy to add you to the list of speakers. 🙂 We are excited to get the ball rolling and make type hints happening. 💪

@StefanBrand
Copy link
Contributor

Almost two months have passed. In the meanwhile I have setup a wiki page in our fork and added commands to generate type stubs using several tools. The resulting stubs can be found in branches in our fork.

Here are all instructions and links: https://github.com/EOX-A/Fiona/wiki/Generating-Python-Stubs

Since using tools to generate type stubs resulted in a big amount of type annotations, I welcome any reviews and comments. The initial focus should be on writerecords() and the collection iterator, as mentioned by @rbuffat above:

Being able to static type check writerecords() or the collection iterator would probably be the most useful thing, [...]

PS.: The lightning talk proposal for FOSS4G 2023 was accepted. So I really need to put some flesh on this. 😁

@StefanBrand
Copy link
Contributor

We are looking for initial feedback on the #1259 draft. Please give your comments. 👌

@StefanBrand
Copy link
Contributor

Cross reference: jazzband/geojson#167

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants