Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add readme to the package API #128

Merged
merged 2 commits into from
Oct 23, 2019
Merged

Add readme to the package API #128

merged 2 commits into from
Oct 23, 2019

Conversation

ruflin
Copy link
Contributor

@ruflin ruflin commented Oct 22, 2019

A package can contain a README.md file for the documentation. If there is a README.md file, it is exposed in the API output for easy grabbing by EPM.

At the moment the user can't configure the default file which should be used, the convention is the README.md file. This file might have an include for further files or data example files.

For now we only support the readme file for documentation. Later on, we might add links to external documentation / repositories if needed like can be seen here: https://doc.rust-lang.org/cargo/reference/manifest.html#package-metadata

An example event with a REAME link looks as following:

{
  "name": "example",
  "title": "Example Integration",
  "version": "1.0.0",
  "readme": "/package/example-1.0.0/docs/README.md",
  "description": "This is the example integration",
  "type": "integration",
  "categories": [
    "logs",
    "metrics"
  ],
  "requirement": {
    "kibana": {
      "version.min": "7.0.0",
      "version.max": ""
    }
  },
  "screenshots": [
    {
      "src": "/package/example-1.0.0/img/kibana-iptables.png",
      "title": "IP Tables Overview dashboard",
      "size": "1492x1382"
    },
    {
      "src": "/package/example-1.0.0/img/kibana-iptables-ubiquity.png",
      "title": "IP Tables Ubiquity Dashboard",
      "size": "1492x1464",
      "type": "image/png"
    }
  ],
  "assets": [
    "/package/example-1.0.0/index.json",
    "/package/example-1.0.0/manifest.yml",
    "/package/example-1.0.0/docs/README.md",
    "/package/example-1.0.0/img/icon.png",
    "/package/example-1.0.0/img/kibana-envoyproxy.jpg",
    "/package/example-1.0.0/elasticsearch/ingest-pipeline/pipeline-entry.json",
    "/package/example-1.0.0/elasticsearch/ingest-pipeline/pipeline-http.json",
    "/package/example-1.0.0/elasticsearch/ingest-pipeline/pipeline-json.json",
    "/package/example-1.0.0/elasticsearch/ingest-pipeline/pipeline-plaintext.json",
    "/package/example-1.0.0/elasticsearch/ingest-pipeline/pipeline-tcp.json",
    "/package/example-1.0.0/kibana/dashboard/0c610510-5cbd-11e9-8477-077ec9664dbd.json",
    "/package/example-1.0.0/kibana/index-pattern/filebeat-*.json",
    "/package/example-1.0.0/kibana/visualization/0a994af0-5c9d-11e9-8477-077ec9664dbd.json",
    "/package/example-1.0.0/kibana/visualization/36f872a0-5c03-11e9-85b4-19d0072eb4f2.json",
    "/package/example-1.0.0/kibana/visualization/38f96190-5c99-11e9-8477-077ec9664dbd.json",
    "/package/example-1.0.0/kibana/visualization/7e4084e0-5c99-11e9-8477-077ec9664dbd.json",
    "/package/example-1.0.0/kibana/visualization/80844540-5c97-11e9-8477-077ec9664dbd.json",
    "/package/example-1.0.0/kibana/visualization/ab48c3f0-5ca6-11e9-8477-077ec9664dbd.json"
  ]
}

@@ -2,6 +2,7 @@
"name": "example",
"title": "Example Integration",
"version": "1.0.0",
"readme": "/docs/README.md",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@neptunian Would that work as an API endpoint? This entry exists, if there is a readme. If there is no README.md file, this entry does not exist and we should fall back to the description.

I wonder if we should call this entry differently like docs_index or similar?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, I like readme for this. We've currently got:

name: essentially an id (e.g. postgres)
title: Short stylized string representation of the package (e.g. PostgreSQL)
description: medium(? not sure how to quantify) length text. Used in our cards for results

We need something for "Long description (pages in length is fine) that allows stylized markup (specifically Markdown)". I think readme works for this.

Authors might reasonably expect docs or documentation to point to their docs site or a page which uses a layout like this image which is common. We wouldn't want something like that on the details page, but it's very common for projects to use this style in documentation.

Screen Shot 2019-10-22 at 5 10 42 PM

Looking at the PostgreSQL 12 docs I don't think we'd want something like this in the details view

Screen Shot 2019-10-22 at 5 15 31 PM

The PostgreSQL README isn't very attractive either, but I think it's more useful and appropriate for the details page.

Begin README paste:

PostgreSQL Database Management System

This directory contains the source code distribution of the PostgreSQL
database management system.

PostgreSQL is an advanced object-relational database management system
that supports an extended subset of the SQL standard, including
transactions, foreign keys, subqueries, triggers, user-defined types
and functions. This distribution also contains C language bindings.

PostgreSQL has many language interfaces, many of which are listed here:

https://www.postgresql.org/download

See the file INSTALL for instructions on how to build and install
PostgreSQL. That file also lists supported operating systems and
hardware platforms and contains information regarding any other
software packages that are required to build or run the PostgreSQL
system. Copyright and license information can be found in the
file COPYRIGHT. A comprehensive documentation set is included in this
distribution; it can be read as described in the installation
instructions.

The latest version of this software may be obtained at
https://www.postgresql.org/download/. For more information look at our
web site located at https://www.postgresql.org/.


End README paste

I realize we not using these repo files specifically, but I think it points to a convention of what people expect in a README and that aligns well with what we're putting in that section of the details view.

There's also precedent in Rust's Cargo.toml manifest format. From their [package] section

# A short blurb about the package. This is not rendered in any format when
# uploaded to crates.io (aka this is not markdown).
description = "..."

# These URLs point to more information about the package. These are
# intended to be webviews of the relevant data, not necessarily compatible
# with VCS tools and the like.
documentation = "..."
homepage = "..."
repository = "..."

# This points to a file under the package root (relative to this `Cargo.toml`).
# The contents of this file are stored and indexed in the registry.
# crates.io will render this file and place the result on the crate's page.
readme = "..."

@ruflin ruflin marked this pull request as ready for review October 22, 2019 20:40
@neptunian
Copy link
Contributor

neptunian commented Oct 22, 2019

Think I'd agree with @jfsiii here. Docs seems like it might be something more extensive rather than pointing to a single markdown file. Although if all the documentation that's going to exist or be needed are in these small readme files then docs would make sense as well.

@ruflin
Copy link
Contributor Author

ruflin commented Oct 23, 2019

Lots of inspiration we can take from the manifest you linked @jfsiii https://doc.rust-lang.org/cargo/reference/manifest.html I think the current state we are is the README.md file. There will be some external links to our docs pages for things that don't fit into the readme. If we really grow to the size of documentation that postgresql has, we will probably need documentation and more. I like also the ides around homepage, authors etc, but I think this comes into play as soon as we have community contributions.

The main difference for the README on our side for now is, that a README can link further files. So my suggestion is we stick for now with readme for now.

A package can contain a README.md file for the documentation. If there is a README.md file, it is exposed in the API output for easy grabbing by EPM.

At the moment the user can't configure the default file which should be used, the convention is the `README.md` file. This file might have an include for further files or data example files.

For now we only support the `readme` file for documentation. Later on, we might add links to external documentation / repositories if needed like can be seen here: https://doc.rust-lang.org/cargo/reference/manifest.html#package-metadata

An example event with a REAME link looks as following:
```
{
  "name": "example",
  "title": "Example Integration",
  "version": "1.0.0",
  "readme": "/package/example-1.0.0/docs/README.md",
  "description": "This is the example integration",
  "type": "integration",
  "categories": [
    "logs",
    "metrics"
  ],
  "requirement": {
    "kibana": {
      "version.min": "7.0.0",
      "version.max": ""
    }
  },
  "screenshots": [
    {
      "src": "/package/example-1.0.0/img/kibana-iptables.png",
      "title": "IP Tables Overview dashboard",
      "size": "1492x1382"
    },
    {
      "src": "/package/example-1.0.0/img/kibana-iptables-ubiquity.png",
      "title": "IP Tables Ubiquity Dashboard",
      "size": "1492x1464",
      "type": "image/png"
    }
  ],
  "assets": [
    "/package/example-1.0.0/index.json",
    "/package/example-1.0.0/manifest.yml",
    "/package/example-1.0.0/docs/README.md",
    "/package/example-1.0.0/img/icon.png",
    "/package/example-1.0.0/img/kibana-envoyproxy.jpg",
    "/package/example-1.0.0/elasticsearch/ingest-pipeline/pipeline-entry.json",
    "/package/example-1.0.0/elasticsearch/ingest-pipeline/pipeline-http.json",
    "/package/example-1.0.0/elasticsearch/ingest-pipeline/pipeline-json.json",
    "/package/example-1.0.0/elasticsearch/ingest-pipeline/pipeline-plaintext.json",
    "/package/example-1.0.0/elasticsearch/ingest-pipeline/pipeline-tcp.json",
    "/package/example-1.0.0/kibana/dashboard/0c610510-5cbd-11e9-8477-077ec9664dbd.json",
    "/package/example-1.0.0/kibana/index-pattern/filebeat-*.json",
    "/package/example-1.0.0/kibana/visualization/0a994af0-5c9d-11e9-8477-077ec9664dbd.json",
    "/package/example-1.0.0/kibana/visualization/36f872a0-5c03-11e9-85b4-19d0072eb4f2.json",
    "/package/example-1.0.0/kibana/visualization/38f96190-5c99-11e9-8477-077ec9664dbd.json",
    "/package/example-1.0.0/kibana/visualization/7e4084e0-5c99-11e9-8477-077ec9664dbd.json",
    "/package/example-1.0.0/kibana/visualization/80844540-5c97-11e9-8477-077ec9664dbd.json",
    "/package/example-1.0.0/kibana/visualization/ab48c3f0-5ca6-11e9-8477-077ec9664dbd.json"
  ]
}
```
@ruflin ruflin changed the title Add Readme in API Add readme to the package API Oct 23, 2019
@@ -2,6 +2,7 @@
"name": "example",
"title": "Example Integration",
"version": "1.0.0",
"readme": "/package/example-1.0.0/docs/README.md",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@neptunian I made a change to the PR that this is now the full path like for all the other assets. Let me know what works best on your end.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, i missed this previously. this is preferable.

@ruflin ruflin merged commit c1c8994 into elastic:master Oct 23, 2019
@ruflin ruflin deleted the readme-in-api branch October 23, 2019 11:52
@ruflin
Copy link
Contributor Author

ruflin commented Oct 23, 2019

@neptunian @skh @jfsiii Related to this change, we discussed if a package can exist without a description. The current answer is no, as each package is validate during startup and having a description is a requirement: https://github.com/elastic/integrations-registry/blob/master/util/package.go#L226

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants