Merge x-pack/docs into docs

[lcawl]

The X-Pack-specific documentation currently resides in https://github.com/elastic/elasticsearch/tree/master/x-pack/docs, but in the long term (now that X-Pack is installed with Elasticsearch), that separation is not required.

The Elasticsearch Reference documentation builds (and navigation tree) will be simpler if we no longer have to pull content from these separate directories.

The following items would need to be addressed:

• Ensuring that X-Pack-related code snippets can be tested successfully after the move. Does this mean that we need a way to flag code snippets that involve non-Basic features? Related to https://github.com/elastic/infra/issues/5608, Docs: Use the default distribution to test docs #31251, QA: Create xpack yaml features #31403, [DOCS] Re-enable code snippet testing for rollup APIs #33319.

• Can we stop generating OSS-only versions of the Elasticsearch Reference? They have not been provided on the web for a while now. If we can cease providing a method to generate OSS-only versions of the book, it avoids the need for complicated include flags. See [DOCS] Removes redundant index.asciidoc files #30707

• Identifying where in the table of contents to integrate the pages that currently exist under the "Setup X-Pack", "X-Pack APIs", and "X-Pack Commands" sections. See [DOCS] Moves X-Pack configuration pages in table of contents #30702

• Addressing broken links and creating redirects for all pages that are deleted or moved.

• Determining whether it is still necessary to highlight pages that contain X-Pack-specific content. Or perhaps only identify information pertaining to non-Basic X-Pack features? Currently, all X-Pack info has an icon next to the page or section title. For example:
  

Add "en" folder under elasticsearch/docs? This currently exists in the elasticsearch/x-pack/docs folder and was added to accommodate translation.

See also elastic/kibana#19156 and elastic/logstash#9596

[elasticmachine]

Pinging @elastic/es-core-infra

[nik9000]

Determining whether it is still necessary to highlight pages that contain X-Pack-specific content.

Personally, I'd like to keep marking things as oss and the license level.

Can we stop generating OSS-only versions of the Elasticsearch Reference?

The last time I tried to do it they didn't work and I don't think anyone had noticed for a few days. As much as I want to keep all the OSS stuff, I think we can probably pitch this.

Ensuring that X-Pack-related code snippets can be tested successfully after the move. Does this mean that we need a way to flag code snippets that involve non-Basic features?

So my plan is to switch the docs build to running the zip distribution (the default one, the one with xpack). After I do that you can just move snippets and it should be fine. I'll see about developing a way run against the oss-zip in a followup change. That'd mean skipping the tests that we can't run based on distribution. My plan would be to scan that information out of the markings that you have at the top of the page to show what APIs require xpack/don't work with the oss distribution.

[nik9000]

So my plan is to switch the docs build to running the zip distribution (the default one, the one with xpack).

I've started work on this. I think you'll be able to do ./gradlew -p docs check and get xpack installed and do ./gradlew -p docs check -Dtests.distribution=oss-zip and get the oss-zip distribution installed. After I'm done with my change both should work. So you can start to port things from the xpack docs to Elasticsearch's docs so long as you pick some consistent way to mark the x-pack only ones. Once you have a few examples of that I should be able to build a thing to skip them if you do -Dtests.distribution=oss-zip.

[lcawl]

@nik9000 Would another option be to have a new string like this under the code example?:
// TEST[level:gold]

[lcawl]

I tested putting "// Comment" strings at the top of the files in #31114 and it was causing doc build errors. It works, however, when I code them as attributes (similar to the current "role"="xpack" attributes). e.g. [testenv="gold+"] We could pick whatever attribute name we wanted, as long as it didn't affect existing attributes that are used by the CSS.

[lcawl]

@nik9000 Just to clarify--do I actually need to specify the license level for testing purposes? Or could all the non-oss tests run on a trial license with all the features enabled? That seems like simpler setup to me.

[nik9000]

I'd prefer you specify the license level though I don't think the tests will use it at first. I'd just like to have it specified so we can automatically stick it on the page and then we can test it at a later date.

[wchrisdean]

[doc issue triage]

[elasticmachine]

Pinging @elastic/es-delivery (Team:Delivery)

[lcawl]

I think it's no longer appropriate to solve the remaining item in the current docs system.