Implicit namespace package support

[uranusjr]

The implicit namespace package is defined in PEP 420 as lacking a __init__.py in the package. But Flit relies on the file to read in the version and (short) description.

Some ways to support this from the top of my head:

1. Add an alternative way to specify those values. Probably against Flit’s philosophy.
2. Add a field in pyproject.toml to instruct Flit to read those values from another file. Implicit namespace packages usually implement a submodule in the parent package, so it’d be possible to have something like top-level = "foo.bar" to instruct Flit to load that module instead of the one defined under the module key.
3. Like the above, but merge this into module. So instead of taking a Python identifier, module can now may also be given a dotted name to specify a submodule. This would make Flit import module for metadata values (as currently), but package all of the top-level module into the distribution. The downside is there’s no way to package multiple submodules into one namespace package in a distribution, but that’s a niche usage anyway (like Flit chose not to support having multiple packages in a distribution).
4. Continue to require a __init__.py in the top-level package, but introduce a flag (e.g. namespace = true) to not package that file into the distribution. I don’t like this approach (the module would behave differently depending on whether it’s imported directly from PWD or after flit install -s), but it’s the easiest hack.

[takluyver]

There's been a PR (#159) hanging around for years; the idea would be that you package one module within the namespace, and flit reads the metadata from that, e.g. sphinxext.opengraph.

I'm just not really interested in namespace packages, so I find it hard to summon the enthusiasm for thinking about how to support them and hearing about bug reports when things go wrong. Quoting myself from that PR:

I ~never use namespace packages myself; my experiences with them have always been that they're an unnecessary complication, creating new ways for things to go wrong for little benefit. So I don't have much motivation to do all the testing to satisfy myself that this will work properly. And I don't have much enthusiasm for maintaining it if it doesn't work.

[maxfischer2781]

Seeing how long the PR has been hanging in the air, is that lack of enthusiasm a show-stopper for adopting namespace packages even if someone else implements/maintains the feature?

Would it be more feasible to have this feature as a third-party, external package that builds on flit? (I will refrain from suggesting to implement it as a namespace package.)

[mivade]

I for one would appreciate namespace package support as my team uses them extensively (I'd prefer if we didn't, but that ship has sailed). Since a lot of my coworkers are not that familiar with all the intricacies of Python packaging, flit would be ideal to use given its simplicity compared to setuptools. Unfortunately I don't have the time to work on this at the moment which is why I'm just adding my voice in favor of supporting namespace packages.

[omry]

I am looking at migrating to flit, and lack of support for namespace packages is forcing me to look for alternatives.

One common use-case for namespace packages is plugins, where multiple separate plugins are contributing to the same namespace independently.

[cjntaylor]

I'm just not really interested in namespace packages, so I find it hard to summon the enthusiasm for thinking about how to support them and hearing about bug reports when things go wrong

That's a shame. Flit almost rescued my floundering project from a lack of direction in poetry, but this leaves me dead in the water. The examples given in your PR (#159) - where a company might have several independent packages all developed under the same, you know, namespace - is far more common than you'd think (often by different teams, so coordinating a single repo that doesn't step on itself is a nightmare at best. Ask me how I know 😉).

One developer to another, you might need to put your feelings to one side here. New-style namespace packages have been a standardized part of the language since 2012. They're not going anywhere. You can whine about them all you want (hey - you of all people should know - write a counter PEP) but, Flit is in the wrong here. You can't expect people to not to use a feature of the language just because you personally don't like or use it (as was pointed out to you several times in that PR).

Now, it's your codebase. You can support (or not support) whatever you'd like. Just know that you're shooting yourself in the foot here with adoption and compatibility with a long standardized aspect of the language. Just see that closed issue from Facebook if you want another less-biased example. Personally, I have no choice but to go back to poetry (which does support namespace packages) and hope I can muddle some horrible workaround together, or find some alternative, like I was trying to do when I dug deeper into flit. This project has so much more potential, too - you're continuing to push out excellent PEPs and really making great strides for the python packaging community as a whole.

On the off chance; how dead is PR #159? It seems ancient by the velocity of this project's commit log - is it even relevant anymore? Would you even consider an updated PR for namespace support? I don't think I really have the time to push it forwards but if there is the potential, I'd consider it.

[maxfischer2781]

The newstyle metadata allows for an explicit version setting. Would it be worthwhile making another proposal for namespace packages (i.e. disregarding #159) that relies on this setting instead of __init__.py?

[martenlienen]

I have implemented support for implicit namespace packages based on new-style metadata in #453 and it is really simple now. The sdist and wheel built for my project look right to me and editable installs work as well.

[takluyver]

OK, I came round to the idea of doing this, even if it's not something I plan to use much myself. I've just released Flit 3.5.

I went back to the idea of #159, where you specify the subpackage or module inside the namespace package (e.g. name = "sphinxcontrib.foo"), not the namespace package itself. That means that getting the version & description from the module still work as with any other module, if you want to use that feature.

This only tries to support PEP 420 'implicit' namespace packages, where the namespace package (the outer one) doesn't have an __init__.py file.

[mivade]

Just wanted to chime in here and say that I've now successfully used this. Thanks @takluyver!