Skip to content

Commit

Permalink
[DOCS] Add data streams to bulk, delete, and index API docs (#58340)
Browse files Browse the repository at this point in the history
Updates existing docs for the bulk, delete and index APIs to make them
aware of data streams.
  • Loading branch information
jrodewig authored Jun 23, 2020
1 parent 433ab42 commit 48f4a8d
Show file tree
Hide file tree
Showing 4 changed files with 65 additions and 26 deletions.
27 changes: 18 additions & 9 deletions docs/reference/docs/bulk.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ POST _bulk

`POST /_bulk`

`POST /<index>/_bulk`
`POST /<target>/_bulk`

[[docs-bulk-api-desc]]
==== {api-description-title}
Expand All @@ -47,8 +47,12 @@ optional_source\n

The `index` and `create` actions expect a source on the next line,
and have the same semantics as the `op_type` parameter in the standard index API:
create fails if a document with the same name already exists in the index,
index adds or replaces a document as necessary.
`create` fails if a document with the same ID already exists in the target,
`index` adds or replaces a document as necessary.

NOTE: <<data-streams,Data streams>> support only the `create` action. To update
or delete a document in a data stream, you must target the backing index
containing the document. See <<update-delete-docs-in-a-data-stream>>.

`update` expects that the partial doc, upsert,
and script and its options are specified on the next line.
Expand All @@ -67,8 +71,8 @@ the `Content-Type` header should be set to `application/x-ndjson`.
Because this format uses literal `\n`'s as delimiters,
make sure that the JSON actions and sources are not pretty printed.

If you specify an index in the request URI,
it is used for any actions that don't explicitly specify an index.
If you provide a `<target>` in the request path,
it is used for any actions that don't explicitly specify an `_index` argument.

A note on the format: The idea here is to make processing of this as
fast as possible. As some of the actions are redirected to other
Expand Down Expand Up @@ -192,8 +196,10 @@ See <<url-access-control>>.
[[docs-bulk-api-path-params]]
==== {api-path-parms-title}

`<index>`::
(Optional, string) Name of the index to perform the bulk actions against.
`<target>`::
(Optional, string)
Name of the data stream, index, or index alias to perform bulk actions
on.

[[docs-bulk-api-query-params]]
==== {api-query-parms-title}
Expand Down Expand Up @@ -321,7 +327,8 @@ operation.
=====
`_index`::
(string)
The index name or alias associated with the operation.
Name of the index associated with the operation. If the operation targeted a
data stream, this is the backing index into which the document was written.

`_id`::
(integer)
Expand Down Expand Up @@ -410,7 +417,9 @@ ID of the shard associated with the failed operation.
`index`::
(string)
The index name or alias associated with the failed operation.
Name of the index associated with the failed operation. If the operation
targeted a data stream, this is the backing index into which the document was
attempted to be written.
======
=====
====
Expand Down
4 changes: 4 additions & 0 deletions docs/reference/docs/delete.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,10 @@ Removes a JSON document from the specified index.
You use DELETE to remove a document from an index. You must specify the
index name and document ID.

NOTE: You cannot send deletion requests directly to a data stream. To delete a
document in a data stream, you must target the backing index containing the
document. See <<update-delete-docs-in-a-data-stream>>.

[float]
[[optimistic-concurrency-control-delete]]
===== Optimistic concurrency control
Expand Down
56 changes: 41 additions & 15 deletions docs/reference/docs/index_.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,27 +6,40 @@

IMPORTANT: See <<removal-of-types>>.

Adds a JSON document to the specified index and makes
it searchable. If the document already exists,
updates the document and increments its version.
Adds a JSON document to the specified data stream or index and makes
it searchable. If the target is an index and the document already exists,
the request updates the document and increments its version.

NOTE: You cannot send update requests for existing documents to a data stream.
To update a document in a data stream, you must target the backing index
containing the document. See <<update-delete-docs-in-a-data-stream>>.

[[docs-index-api-request]]
==== {api-request-title}

`PUT /<index>/_doc/<_id>`
`PUT /<target>/_doc/<_id>`

`POST /<index>/_doc/`
`POST /<target>/_doc/`

`PUT /<index>/_create/<_id>`
`PUT /<target>/_create/<_id>`

`POST /<index>/_create/<_id>`
`POST /<target>/_create/<_id>`

[[docs-index-api-path-params]]
==== {api-path-parms-title}

`<index>`::
(Required, string) Name of the target index. By default, the index is created
automatically if it doesn't exist. For more information, see <<index-creation>>.
`<target>`::
(Required, string) Name of the data stream or index to target.
+
If the target doesn't exist and matches the name or wildcard (`*`) pattern of a
<<create-a-data-stream-template,composable template with a `data_stream`
definition>>, this request creates the data stream. See
<<set-up-a-data-stream>>.
+
If the target doesn't exist and doesn't match a data stream template,
this request creates the index.
+
You can check for existing targets using the resolve index API.

`<_id>`::
(Optional, string) Unique identifier for the document. Required if you are
Expand All @@ -48,6 +61,9 @@ if it does not already exist (_put if absent_). If a document with the specified
`_id` already exists, the indexing operation will fail. Same as using the
`<index>/_create` endpoint. Valid values: `index`, `create`.
If document id is specified, it defaults to `index`. Otherwise, it defaults to `create`.
+
NOTE: If the request targets a data stream, an `op_type` of `create` is
required. See <<add-documents-to-a-data-stream>>.

include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=pipeline]

Expand Down Expand Up @@ -125,12 +141,19 @@ You can index a new JSON document with the `_doc` or `_create` resource. Using
exist. To update an existing document, you must use the `_doc` resource.

[[index-creation]]
===== Create indices automatically
===== Automatically create data streams and indices

If request's target doesn't exist and matches a
<<create-a-data-stream-template,composable template with a `data_stream`
definition>>, the index operation automatically creates the data stream. See
<<set-up-a-data-stream>>.

If the specified index does not already exist, by default the index operation
automatically creates it and applies any configured
<<indices-templates,index templates>>. If no mapping exists, the index operation
creates a dynamic mapping. By default, new fields and objects are
If the target doesn't exist and doesn't match a data stream template,
the operation automatically creates the index and applies any matching
<<indices-templates,index templates>>.

If no mapping exists, the index operation
creates a dynamic mapping. By default, new fields and objects are
automatically added to the mapping if needed. For more information about field
mapping, see <<mapping,mapping>> and the <<indices-put-mapping,put mapping>> API.

Expand All @@ -142,6 +165,9 @@ automatic creation of indices that match specified patterns, or set it to
comma-separated list of patterns you want to allow, or prefix each pattern with
`+` or `-` to indicate whether it should be allowed or blocked. When a list is
specified, the default behaviour is to disallow.
+
IMPORTANT: The `action.auto_create_index` setting only affects the automatic
creation of indices. It does not affect the creation of data streams.

[source,console]
--------------------------------------------------
Expand Down
4 changes: 2 additions & 2 deletions docs/reference/rest-api/common-parms.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -430,8 +430,8 @@ end::index-total[]
tag::bulk-index[]
`_index`::
(Optional, string)
The name of the target index.
Required if not specified as a path parameter.
Name of the data stream, index, or index alias to perform the action on. This
parameter is required if a `<target>` is not specified in the request path.
end::bulk-index[]

tag::index-metric[]
Expand Down

0 comments on commit 48f4a8d

Please sign in to comment.