What categories should be available on crates.io? [BIKESHED ALERT 🚲🏠🚨] #488
Conversation
carols10cents
force-pushed
the
proposed-categories
branch
from
94c1742 to
85ff1fe
Compare
|
My first thought reading through is that there's a hierarchy of structured data. Representing that in a line-oriented manner is sub optimal. There's repetition of aspects like the English text for each category that seems brittle. I also am confused what would go in a category like Some hypothetical JSON including these ideas: [
{
"slug": "libraries",
"title": "Libraries",
"description": "Libraries are the powerhouse of the cell",
"children": [
{
"slug": "astronomy",
"title": "Astronomy",
"description": "Astronomy is the study of the things 'up there'. These libraries help in that pursuit"
}
]
}
]Although I'd expect it to be TOML, really 😇 |
I like the JSON, especially with the ability to add a description. Should it be TOML tho? :) [[categories]]
slug = "libraries"
title = "Libraries"
description = "Libraries are the powerhouse of the cell"
[categories.children]
slug = "astronomy"
title = "Astronomy"
description = "Astronomy is the study of the things 'up there'. These libraries help in that pursuit"The current text representation came from me starting out with just the display name for each category per line, then @alexcrichton suggested adding slugs in a simple way. Alex, wdyt about switching to JSON or TOML? (I know you're at #mozloha, I don't expect a quick response, but I don't want to put the work into doing this until you have a chance to weigh in ❤️ ) |
|
Ah this looks cool! I've got two thoughts/questions. :-) First, can a crate belong to more than one category? If the answer is yes, then I feel like this makes it more like a tag concept than a categorical one, but that's just my own perspective. I don't know if others feel the same way. Second...
I feel like this is a reasonable starting place. But let's say a year from now we wanted to change our minds and expand our categories into things other than just topic classifiers. Would that be feasible to do? |
Yes, up to 5.
To me, the current keyword functionality is more like tags, for the reasons I have in the description under the bullet point "Categories are different from keywords because:". Can you elaborate a bit on what the difference is, to you, between keywords and categories and tags, and how to best communicate that?
Minimally, we could add additional categories with different semantics to this file, perhaps by adding some "top level" categories to distinguish them. I would not recommend adding the top-level "Topic" category like PyPI has at that point, since everyone would need to update their Cargo.tomls. I suppose we could just change the display name of existing categories to add "Topic::" or similar, with the understanding that the slugs wouldn't have that (and potentially be confusing). |
Ah I see. I guess I feel like tags express a many-to-many relationship where as categories express a one-to-many relationship. I think keywords is the right word for what we have now, because they express a many-to-many relationship and are not curated. I guess I see tags as things that are curated, but also express many-to-many relationships. This is all kind of subjective though. I think you might have been coming at it from the curated/uncurated perspective where as I'm coming at from the relationship perspective. No clue which one is right!
Ah cool that seems reasonable to me! |
Oh, thought of another thing we might want to add if we have categories for multiple dimensions... the way you navigate through classifiers on PyPI is that you can select any number of them to find libraries in the intersection. Try it out! One of the reasons I'm not so much a fan of this approach is that I'd want the ability to filter searches for some of these attributes as well, not just while browsing through topic categories (ex: search for "piston", which isn't a category, and limit to crates with windows support only), and it doesn't appear that PyPI lets you do that. IMO it'd be awesome if we had sorting/filtering mechanisms that worked for both browsing and searching. |
|
Is this the good place to suggest categories? If yes, I'd like to suggest "(libraries::)internationalization". (Also, not sure I get why there are separate categories for applications and libraries instead of e.g. having a separate flag?) |
Yes it is!! Adding that one now :)
I could be argued out of it :) I mostly did that because that's what Awesome Rust did! |
|
One potential ambiguity: do the various If those categories have one of the broader meanings, I'd suggest widening the names from |
|
I have mixed feelings about this. I do think that better discovery mechanism is definitely needed — currently homepage features very little content ("most downloaded" crates are boring), and exploring crates by alphabetically sorted names doesn't really work. I'm a little bit worried about adding predetermined categories. This seems like a top-down decision strongly suggesting what Rust is intended to be used for. It's a change from a bottom-up approach of a soup of tags where anything goes. I worry it may be discouraging writing something that doesn't fit any category, or that is in a category that's much smaller than other categories. Maybe categories could be built from most popular tags? (after cleanup of synonyms) Being able to choose multiple categories is weird. To me it makes sense to have one primary category + keywords. Otherwise it overlaps with tags too much, especially when anybody can put their package into a category (i.e. categories are just another form of tag search, rather than curated collections/showcases). It'd make sense to me if categories were tags, i.e. packages tagged with a category-related tag appear in that category. |
|
The It might make sense to have a category for type of UI: I feel like Finally, I think the existing categories need to distinguish between "graphics" (which should include things like image formats and drawing libraries) and "gui" (which should include things like toolkits and user-interface libraries). |
|
Now, the bikeshed:
|
|
Let's bikeshed !
I see the point of
How do we decide which language gets in this list of categories? For example I see both mruby and ruby, but no micro-python. C++ as a well known language is missing too, Objective-C is there but not Swift or Go.
I would unify these two categories in a Having subcategories (
These two could be unified too.
As @shepmaster, I do not see what would fit in there.
I feel the same, for example my next project to be released on crates.io will be both an application and a library, plus the extra |
|
Also, we could maybe use the most used tags on crates.io as a starting point for the category list. |
|
|
Maybe categories could be built from tags the way StackOverflow does it? Every tag search page has "Related tags" in sidebar that narrows down the search: https://stackoverflow.com/questions/tagged/rust That seems to automatically create a good hierarchy. |
Here's all the keywords. Top 20, would any of these be good categories that aren't already covered here?
I didn't start from the keywords because I personally don't find the current keywords useful at all, and it's mostly because of the "anything goes" approach. For example, crates with the CLI keyword include argonaut, "A simple argument parser" to help you build CLIs, and betsey, "An AppVeyor cli written in Rust", which is an application with a CLI for use with a particular tool, appveyor. And technically this is an addition, not a change. Adding categories is not getting rid of keywords, and people can add both to their crates and use either to find crates they're looking for.
I get what you're saying here, and I don't want to discourage anyone. Perhaps adding some documentation to the "All Categories" page about how to send a PR to suggest a new category would help? Many other package websites have categories in ways similar to this, and I haven't experienced or heard of any problems like this. Have you? I'd be really interested, especially if there are any blog/forum posts discussing this from other communities! I will be addressing the actual category bikeshed comments in a bit, wanted to speak to some of the meta-comments first :) |
|
libpng and Unreal Engine would probably both end up in Graphics category, which doesn't feel right (there's no specific category for 3d engines or image encoding). |
|
Just my 2c. I wish we could have some community-driven tag-based system. Eg. Maybe in This way crates.io could use it as "votes", and each popular crate would get described by people that actually use it. Plus tags like that serve as a nice documentation of dependencies. Wild idea, I know. |
|
Categories are a lot like OO class hierarchies, and I think the problems there will carry across (diamond inheritance and being unable to find a good structure that every crate can fit into). I think it could be OK as long as the hierarchies are kept relatively shallow. What if within each category, crates.io could show "pseudo sub-categories" based on the most common tags within those categories? For example, instead of trying to list up-front all the sub-categories for "dev tools", explicit categorisation could stop there, and then within "dev tools", crates.io would look at the common tags within that category (eg. "testing") and show those as the next layer of subcategories. At some point later those common tags could be promoted into a full subcategory. It could choose tags to show based on what fraction of the crates have the tag - ideal tags would be those where approximately 1/n crates have the tag, where "n" is the desired branching level of categories, say 10. The tags which come closest to this ratio would be shown as pseudo-subcategories (listed after any actual subcategories). This relies on being able to put crates into categories which are not leaf nodes on the tree of categories - is that possible? I think it should be possible to put a crate directly in the "Libraries" category without putting it in one of the sub-categories. |
|
Something I want to chime in about categories. As a crate author, I have hard time picking out categories. Mostly because I’m a lazy goon and dislike running through a list of ten hundred strings to pick five best fitting to my crate. This probably signifies that I’m much more comfortable with a list of ten (not tens of hundreds) very generic categories from which I pick one. Whatever way this feature goes, I’ll probably end up not using it, but having less categories rather than more will certainly make me more prone to the appeals of categories :) |
|
I agree with nagisa; too many categories defeats the purpose. I think that categories should not be added, and instead more emphasis should be placed on tags. Differentiating between libraries and applications is something that would be very useful. Perhaps a more advanced search allowing for filtering of libs/applications and easier searching of tags. We already have tags as a part of the |
|
I have a file size humanization library, I don't think it fits in an existing category, and I'm not sure what to suggest. |
|
What many sites do is call everything a "tag", but distinguish between a small list of "official" tags that have a defined hierarchy and descriptions, and arbitrary "freeform" tags. Both would appear in the same tag list within crates, but the site would prominently show official tags with descriptions, while using freeform tags just for search. |
|
I think tags (I think of tags and keywords as the same thing) and a better search would be better than categories. 2 issues looking at that screnshot:
If we end up having categories, I'd agree with @BurntSushi and have only one possible category. The fact that categories are curated is not a big enough reason to have several imo. If we have categories, I'd imagine something like (didn't list possible categories): bin:
- games
- audio etc
library:
- ffi
- science
- web
- cli
- graphics
- gamedev
- crypto
- database
- markup etcIn short:
Category should just be there to give a general idea, we have keywords for filtering down. Aside: Is the full crate.io data available (ie with description, tags etc)? The index only seems to contain what cargo needs. |
|
I'm missing categories for several of my crates:
Will there be descriptions and/or a manual to go along with the categories? For example I think it'd be important to note that libraries::hashing is not libraries::cryptography. And I'd like some guidance for where to put my library-example crates (same as library? applications? development tools?). |
I think If I were looking for a library to help me FFI with, say, Ruby, I probably wouldn't have a preconception necessarily about whether a binding generator/FFI helper library or a binding to libraries of those languages would be what I wanted, so I think the categories should have a broader meaning. Anyone who has more experience searching for FFI related crates, please chime in if this isn't the case for you! I think I'm just going to move the FFI categories beneath "Libraries" for now, since most of the tools I know of integrate with your code in a library fashion somehow. Are there any purely external applications that do, say, binding generation? |
Multimedia sounds good :)
I'm a little unsure about this one. I'm trying to imagine going to crates.io and saying "I really want an application that has a GUI today, to do X". I feel like I'd just want to do X and not particularly care about the interface. Can you give an example of a case where these categories would be useful?
Adding Libraries::Ownership, wdyt? Any others?
Added Libraries::GUI; this is definitely one that a description would help with. |
Since that fits into algorithms.
Removed some categories we found it hard to justify.
I don't think that necessarily follows. I think the guidance of what categories are available will be enough for crate authors to categorize their crates in a useful fashion; once this PR is merged, @shepmaster and I plan to send PRs to popular crates to help seed the categories with crates. If not the crate authors, who do you propose should do this work?
We've considered adding a command to cargo to update metadata only; as we add more metadata, that command becomes more important, definitely, it's just that no one has implemented it yet. |
carols10cents
force-pushed
the
proposed-categories
branch
from
ff836db to
2c1b405
Compare
|
Ok, I decided I've been convinced to remove the top-level Application and Library categories (but I kept the top level Developer Tools category). I moved all the subcategories up to the top level and merged them together where appropriate. The thing that tipped my decision is that now on /categories, you see 37 categories available, instead of only 3 (Applications, Developer Tools, and Libraries). I think this makes it clearer, sooner, the breadth of all the different kinds of crates available and makes a better first impression. Here's a preview:
I'd like to get a general sense of what everyone who's contributed to this issue thinks of this late, big change, if you have a minute please add a reaction to this comment:
|
@LukasKalbertodt In the last commit when I was merging the libraries and applications together, I also added "API bindings" ("Wrappers of specific APIs for convenient access from Rust. Includes |
|
🌈 @carols10cents Having only three categories shown at the top level ( It seems like the main reason for your decision to flatten out the tree is a visual one. Why not fix this by displaying the categories in a different manner instead of removing semantics from the category-tree? For example, we could have a special rendering rule for |
That was only the last reason for my decision :) The other factors include:
Relevant comments: |
|
If no one comments on this today, I'm going to say it's done... |
|
Whew. Ok @alexcrichton, I think we're at least out of objections to the current color of the bikeshed :) |
|
Awesome, thank for pushing this through @carols10cents! |
|
Err I just realized that it's impossible to ever remove a category, since crates put the category name in their Cargo.toml. Now I regret not continuing the discussion about the huge imbalance between gamedev and non-gamedev categories. |
It is not impossible: crates that have invalid categories in their Cargo.toml will still get published, just with a warning in the output of Or was there something else you mean by "impossible"? |
No, that's what I was thinking of. If categories can be changed then it's ok to see in practice which categories will be full and need to be split, which categories will be empty and could be removed, and which categories are missing. |
TL;DR
Categories are coming to a crates.io near you! Help decide what categories should be made available!
Take a look at the diff and consider:
As a crate user:
As a crate author:
Details
::indicates a sub-category relationship. See screenshots for how subcategories will work.