Improve Beats field reference docs to reflect ECS

[dedemorton]

Describe the enhancement:
The Beats field reference docs (for example, Exported fields ) were put together before ECS came along. They need to be adapted to make sense in the context of ECS.

We need to fix this programmatically because updating the individual field.yml files manually would be crazy making. I also think the field descriptions should be single-sourced with the ECS docs rather than duplicated.

Describe a specific use case for the enhancement or feature:
When users want to understand what an event field contains, they might turn to the docs. However, it's not always easy to find field descriptions because the docs use the older field names rather than the ECSs name. Often the ECS names appear in the description. For example:

This is hard on users for a couple of reasons:

• When they scan the page, they can't find the field easily.
• It's not clear what "alias to" even means.
• There's no mention of ECS or link to the ECS docs.

We should show the ECS name. I'm not even sure it still makes sense to show the older field names, especially since the aliases are going away in 8.0.

[elasticmachine]

Pinging @elastic/integrations (Team:Integrations)

[dedemorton]

@webmat Is this a fair description of the problem?

[mostlyjason]

Hopefully this won't be a problem in the new integration README docs. Those will be separate from Beats but good to keep in mind

[dedemorton]

@mostlyjason That's right!

[webmat]

Yeah I think these module docs about which fields are populated by this module could be improved in a few ways. Hopefully the upcoming packages are already doing something like this:

• List all fields that are populated by this data source, no matter if it's an ECS field or a custom field. Globs may be acceptable (e.g. source.geo.*)
• For each field, the doc should indicate whether or not it's an ECS field
• Finally, the alias fields you highlight here were part of the 6 to 7 migration. They're going away in 8. So I would treat those differently. I think we could have a special section at the end of these pages that lists all fields that were migrated, with a mention that these will be going away.

[mostlyjason]

I just checked out the integration guidelines and it looks like we have a logs and metrics field reference we intend on keeping up to date. @dedemorton did you give input on the guidelines and field reference docs already?

[dedemorton]

@mostlyjason Yes, but @EamonnTP is more involved with the docs related to the UI.

The problem I'm trying to describe in this issue is related to our legacy Beats documentation that hasn't been updated to reflect the ECS names. It's confusing for users to see the old names in the Beats reference docs.

[bmorelli25]

A quick note: there's a related discussion about APM fields in elastic/apm#276, with an interim solution proposed in #21359.

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.