[ML] adds new question_answering NLP task for extracting answers to questions from a document

[benwtrent]

This commit adds a new question_answering task.

The question_answering task allows supplying a question in the inference config update.

When storing the model config for inference:

"inference_config": {
  "question_answering": {
    "tokenization": {...}, // tokenization settings, recommend doing 386 max sequence length with 128 span, and no truncating
    "max_answer_length": 15 // the max answer length to consider
  }
}

Then when calling _infer or running with in a pipeline, add the question you want answered given the context provided by the document text

{
  "docs":[{ "text_field": <some long text field to extract answer}],
  "inference_config": {
    "question_answering": {
      "question": <Question desiring answer>
    }
  }
}

The response then looks like:

{
    "predicted_value": <string subsection of the document that is the answer>
    "start_offset": <Char offset in document to start>,
    "end_offset": <char offset end of the answer,
    "prediction_probability": <prediction score>
}

Some models tested:

• https://huggingface.co/distilbert-base-cased-distilled-squad
• https://huggingface.co/bert-large-uncased-whole-word-masking-finetuned-squad
• https://huggingface.co/deepset/electra-base-squad2
• https://huggingface.co/deepset/tinyroberta-squad2

[elasticmachine]

Pinging @elastic/ml-core (Team:ML)

[elasticsearchmachine]

Hi @benwtrent, I've created a changelog YAML for you.

[davidkyle]

Nits, nothing more.

LGTM I added the cloud-deploy label so I can test it out

[davidkyle]

not sure what is happening here

[benwtrent]

We need to verify that the end is actually after the start. If we don't we could get into a death spiral where the splitting never moves forward (think adversarial case of very small sequences with overlaps that are almost the same as the sequence size).

[davidkyle]

Good fix

[davidkyle]

This looks very similar to

tokenize( String seq1, InnerTokenization innerResultSeq1, String seq2, Tokenization.Truncate truncate, int sequenceId )

but with the spanning logic added. Is there an opportunity to refactor here?

[benwtrent]

Maybe, but the previous one shouldn't do spanning at all. Mainly because its an optimization to allow a single tokenization to be used multiple times. We just have no way of representing a spanned batched encoding (4d tensor?).

I think some refactoring could be done as a whole, but it would take more churn on the rest of the tokenization code.

[benwtrent]

@elasticmachine update branch

[benwtrent]

@elasticmachine update branch

[benwtrent]

@elasticmachine update branch