How should modules and themes be evaluated for inclusion?

[phenaproxima]

Right now, we are in a rapid prototyping phase of developing this project. That means, effectively, that whatever I like, is what goes in.

As fast as that approach lets us be, we can't do it forever. Once this project is released to the world, we're going to need some kind of explicit process and policy around including (or excluding) modules and themes.

I discussed this with larowlan in Slack and here were some thoughts which came out of that, in no particular order:

• Figuring this out is definitely a GA blocker.
• I think there should be a "selectboard" of sorts - a group of folks who have to agree by consensus about including a particular module, theme, or recipe in this project.
• The consensus process must be written out in an issue or PR.
• I think that the selectboard should consist of one core product manager (to consider value), one core release manager (to consider stability), one developer with security team membership (to consider code security), one person on the UX team, and one experienced site builder who doesn't code. Each of these people must have worked on at least one real-world client project per year to retain their membership in the selectboard.
• Code quality is probably not a terribly important consideration for this project. Security and test coverage are factors, but we should consider ease of use, value-to-complexity, number of installs, how active and available the maintainers are, and so forth.

[larowlan]

I think this should also go on the list

• Stability of the data model and conformance to standard Drupal data model patterns.

E.g use of the correct storage model - storing the right stuff in state/config/entities/third party settings

Have seen some modules that store stuff that belongs in state in config because 'config is variable_get from d7'

There's also a value judgement around the data model when it comes to upgrade paths. E.g. I've seen some presentation modules that use paragraph content entities to store what should be deployable configuration.

In the case of config, should have the correct dependencies too

[phenaproxima]

Some possible criteria for inclusion:

• Active maintainership -- that means at least one maintainer needs to be available for questions and bug fixes, and is on top of keeping the extension compatible with the latest releases of Drupal in a timely manner -- i.e., once core releases a new major beta version, they should be working to make the extension compatible. Willingness to work and communicate with the Starshot maintenance "crew" is also essential here.
• Battle-tested -- a high number of installs (> 10K) and/or being vouched for by multiple users/agencies, should count for something. Being a long-running project should also count in a module's favor.
• As @larowlan pointed out, conformance to standard data modeling paradigms. Modules that abuse or distort Drupal's data modeling are likely to cause maintenance problems down the road. (This criterion is a bit of a slippery slope; it could be argued that essential things like Entity Reference Revisions fall on the wrong side of the line, so I think this is a judgment call that should be made by an experienced core developer or committer.)
• Update path and API stability. Breaking backwards compatibility in patch releases, or not providing update paths, should be a very big red flag. Modules with active "ecosystems" are likely to be safer in this regard.

[japerry]

Alongside this, there should be better documentation / architecture diagrams stating the criteria in which modules meet requirements for inclusion. As @phenaproxima flagged, while ERR is a popular module, it violates the atomic entity data model for Drupal, and thus should be avoided. There are other 'best practices' that need to be documented, like minor/major version support, making sure modules don't inadvertently break php requirements, etc.

Also, sorta obvious but needs to be opted into security advisories.

[phenaproxima]

There a lot of factors. A module like ERR is a problem case because even though it does questionable things at the data layer, it enables (via Paragraphs) so many other critically important uses that the risk might be outweighed by the benefits for people who need component-based page building systems (i.e., pretty much everyone). Absent a better solution, that is...which is where the Experience Builder initiative comes in, but it may be a while before that's viable for inclusion in Starshot. In the meantime we might have to fall back on Paragraphs, at least for now, to deliver something useful.

To be clear, I'm not the one who's ultimately going to make these decisions, but I think I'd definitely advocate for "immediate usefulness" over correctness. It is NOT, as far as I know, within Starshot's scope promise any kind of upgrade, crossgrade, or migration paths beyond what's provided by core and contributed modules. So we can change our page building solution whenever it makes sense to do so.

[simesy]

I think these requirements should be driven by the product owner of starshot. I'm going to assume that is Dries because of the comments in the keynote about clearing his schedule. What features does Dries want out of the box?

A different approach ... where/when are the personas defined?

The abilty to install more recipes should be left to the project browser team, and that effort funnelled into how additional recipes are made available. So if none of the "out of the box experience" personas will need an Event, then it doens't need to be in scope here.

[simesy]

To be clear, I'm not the one who's ultimately going to make these decisions

Is there a timeline for when this person is appointed?

It is NOT, as far as I know, within Starshot's scope promise any kind of upgrade, crossgrade, or migration paths beyond what's provided by core and contributed modules.

I'm banking on this! I think this is clear.

[phenaproxima]

Is there a timeline for when this person is appointed?

That, I don't know.

But I don't think it will be, or should be, one person. I think it needs to be a small group of people (like around 4) who regularly build sites for clients, and who are product-oriented and end user-oriented. I think technical correctness should be a relatively low priority in Starshot; the focus should be on solving the problems faced by authors, site builders, and non-technical Drupal users, as quickly as possible, using the best practices available to us.

Honestly, I'm hoping I can maybe chat with Dries and @goba to put some of these thoughts in front of them. I think Starshot really should have a written list of "values", or guiding principles, to keep this project on the right track. And I agree with you that the personas we're aiming at should also be clearly defined.

[simesy]

think it needs to be a small group of people (like around 4) who regularly build sites for clients, and who are product-oriented and end user-oriented.

Yeah i like it. I think in my mental map there are two functions.

• what to build
• how to build it

They could be same or different people for both of these things. But i think "What to build" is more subjective and will be more end user focussed (client) focussed, whereas "How to build" might be more UX/DX/Security focussed.

Most imporantly "what to build" should come first :)

[simesy]

(I'm not saying we don't (as drupal community) have a pretty good idea of what is good and bad, but we want a polished product not a race to fill a bucket with features)