Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TA] Add analyze tasks feature support #17267

Merged
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
7b0e24c
TA-Healthcare
mssfang Nov 5, 2020
446bb42
remove checkstyle supppresion
mssfang Nov 5, 2020
5ac3e30
regenerate swagger ater a new change merged in
mssfang Nov 5, 2020
c03aa75
transfer to laptop
mssfang Nov 5, 2020
968acb9
cancellation is working now
mssfang Nov 6, 2020
1b5dbc8
address mari's most feedbacks
mssfang Nov 6, 2020
53e0d0e
replace a wrong json file for async pagination test
mssfang Nov 6, 2020
53654d6
init analyze tasks
mssfang Nov 6, 2020
8d79350
removed xxxTaskState class and TaskStete
mssfang Nov 6, 2020
5c7f4ce
add changelog
mssfang Nov 6, 2020
8c57c6c
resolve conflict and updates changes
mssfang Nov 6, 2020
37c3bf1
resolved conflict and add more tests
mssfang Nov 11, 2020
febe0be
regenerate code base on swagger 5ef5a597b3f2342bfd254ed79b97b2fe160e50a1
mssfang Nov 13, 2020
9a25060
address feedbacks
mssfang Nov 13, 2020
ee4ae09
improve PLAYBACK test speed
mssfang Nov 13, 2020
cd58ae8
checekstyle issue
mssfang Nov 13, 2020
d1c02b5
update Analyze API endpoint
mssfang Nov 17, 2020
c94420c
add issue link to TODO lists
mssfang Nov 17, 2020
8802740
renaming some APIs and address feedbacks
mssfang Nov 18, 2020
ae88549
make final class if possible
mssfang Nov 18, 2020
047f2b5
make HealthcareEntityCollection final class
mssfang Nov 18, 2020
201e025
service already fixed the nextLink to @nextLink
mssfang Nov 18, 2020
d52decd
add NPE and add live test setup
mssfang Nov 19, 2020
668650b
address last feedbacks
mssfang Nov 19, 2020
b819e4f
update readme content and links
mssfang Nov 19, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion sdk/textanalytics/azure-ai-textanalytics/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
# Release History
## 5.1.0-beta.3 (Unreleased)
**New features**
samvaity marked this conversation as resolved.
Show resolved Hide resolved
- Added support for Healthcare analysis, it is a long-running operation, and the cancellation supported.
- Added support for healthcare analysis feature, it is a long-running operation, and the cancellation supported.
- Added support for analyze tasks feature, It analyzes multiple tasks, such as, entity recognition, PII entity recognition
and key phrases extraction simultaneously in a list of document.
mssfang marked this conversation as resolved.
Show resolved Hide resolved

## 5.1.0-beta.2 (2020-10-06)
### Breaking changes
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,7 @@ private PagedResponse<HealthcareTaskResult> toTextAnalyticsPagedResponse(

final HealthcareTaskResult healthcareTaskResult = new HealthcareTaskResult(
// TODO: change back to UUID after service support it.
healthcareJobState.getJobId().toString(),
healthcareJobState.getJobId(),
healthcareJobState.getCreatedDateTime(),
healthcareJobState.getLastUpdateDateTime(),
toJobState(healthcareJobState.getStatus()),
Expand Down

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,15 @@
import com.azure.ai.textanalytics.implementation.TextAnalyticsClientImpl;
import com.azure.ai.textanalytics.models.AnalyzeSentimentOptions;
import com.azure.ai.textanalytics.models.AnalyzeSentimentResult;
import com.azure.ai.textanalytics.models.AnalyzeTasksOptions;
import com.azure.ai.textanalytics.models.AnalyzeTasksResult;
import com.azure.ai.textanalytics.models.CategorizedEntityCollection;
import com.azure.ai.textanalytics.models.DetectLanguageInput;
import com.azure.ai.textanalytics.models.DetectLanguageResult;
import com.azure.ai.textanalytics.models.DetectedLanguage;
import com.azure.ai.textanalytics.models.DocumentSentiment;
import com.azure.ai.textanalytics.models.HealthcareTaskResult;
import com.azure.ai.textanalytics.models.JobManifestTasks;
import com.azure.ai.textanalytics.models.KeyPhrasesCollection;
import com.azure.ai.textanalytics.models.LinkedEntityCollection;
import com.azure.ai.textanalytics.models.PiiEntityCollection;
Expand Down Expand Up @@ -78,6 +81,7 @@ public final class TextAnalyticsAsyncClient {
final RecognizePiiEntityAsyncClient recognizePiiEntityAsyncClient;
final RecognizeLinkedEntityAsyncClient recognizeLinkedEntityAsyncClient;
final AnalyzeHealthcareAsyncClient analyzeHealthcareAsyncClient;
final AnalyzeTasksAsyncClient analyzeTasksAsyncClient;

/**
* Create a {@link TextAnalyticsAsyncClient} that sends requests to the Text Analytics services's endpoint. Each
Expand All @@ -101,6 +105,7 @@ public final class TextAnalyticsAsyncClient {
this.recognizePiiEntityAsyncClient = new RecognizePiiEntityAsyncClient(service);
this.recognizeLinkedEntityAsyncClient = new RecognizeLinkedEntityAsyncClient(service);
this.analyzeHealthcareAsyncClient = new AnalyzeHealthcareAsyncClient(service);
this.analyzeTasksAsyncClient = new AnalyzeTasksAsyncClient(service);
}

/**
Expand Down Expand Up @@ -1060,4 +1065,34 @@ public PollerFlux<TextAnalyticsOperationResult, PagedFlux<HealthcareTaskResult>>
public PollerFlux<TextAnalyticsOperationResult, Void> beginCancelAnalyzeHealthcare(UUID jobId) {
return analyzeHealthcareAsyncClient.beginCancelAnalyzeHealthcare(jobId, Context.NONE);
}

// Analyze LRO
mssfang marked this conversation as resolved.
Show resolved Hide resolved
/**
* Analyze tasks, such as, entity recognition, PII entity recognition and key phrases extraction in a list of
* {@link TextDocumentInput document} with provided request options.
*
* See <a href="https://aka.ms/talangs">this</a> supported languages in Text Analytics API.
*
* <p><strong>Code Sample</strong></p>
* {@codesnippet com.azure.ai.textanalytics.TextAnalyticsAsyncClient.beginAnalyze#Iterable-String-JobManifestTasks-AnalyzeTasksOptions}
*
* @param documents A list of {@link TextDocumentInput documents} to be analyzed.
* @param displayName Job's display name.
* @param jobManifestTasks A collection of analyze tasks, such as entity recognition, PII, and key phrase tasks.
* @param options The additional configurable {@link AnalyzeTasksOptions options} that may be passed when
* analyzing a collection of tasks.
*
* @return A {@link PollerFlux} that polls the analyze a collection of tasks operation until it has completed,
* has failed, or has been cancelled. The completed operation returns a {@link PagedFlux} of
* {@link AnalyzeTasksResult}.
*
* @throws TextAnalyticsException If analyze operation fails.
* @throws NullPointerException If {@code jobId} is null.
mssfang marked this conversation as resolved.
Show resolved Hide resolved
*/
@ServiceMethod(returns = ReturnType.COLLECTION)
public PollerFlux<TextAnalyticsOperationResult, PagedFlux<AnalyzeTasksResult>> beginAnalyze(
mssfang marked this conversation as resolved.
Show resolved Hide resolved
Iterable<TextDocumentInput> documents, String displayName, JobManifestTasks jobManifestTasks,
AnalyzeTasksOptions options) {
return analyzeTasksAsyncClient.beginAnalyze(documents, displayName, jobManifestTasks, options, Context.NONE);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,15 @@
package com.azure.ai.textanalytics;

import com.azure.ai.textanalytics.models.AnalyzeSentimentOptions;
import com.azure.ai.textanalytics.models.AnalyzeTasksOptions;
import com.azure.ai.textanalytics.models.AnalyzeTasksResult;
import com.azure.ai.textanalytics.models.CategorizedEntity;
import com.azure.ai.textanalytics.models.CategorizedEntityCollection;
import com.azure.ai.textanalytics.models.DetectLanguageInput;
import com.azure.ai.textanalytics.models.DetectedLanguage;
import com.azure.ai.textanalytics.models.DocumentSentiment;
import com.azure.ai.textanalytics.models.HealthcareTaskResult;
import com.azure.ai.textanalytics.models.JobManifestTasks;
import com.azure.ai.textanalytics.models.KeyPhrasesCollection;
import com.azure.ai.textanalytics.models.LinkedEntity;
import com.azure.ai.textanalytics.models.LinkedEntityCollection;
Expand Down Expand Up @@ -925,4 +928,38 @@ public SyncPoller<TextAnalyticsOperationResult, PagedIterable<HealthcareTaskResu
public SyncPoller<TextAnalyticsOperationResult, Void> beginCancelAnalyzeHealthcare(UUID jobId, Context context) {
return client.analyzeHealthcareAsyncClient.beginCancelAnalyzeHealthcare(jobId, context).getSyncPoller();
}

// Analyze

/**
* Analyze tasks, such as, entity recognition, PII entity recognition and key phrases extraction in a list of
* {@link TextDocumentInput document} with provided request options.
*
* See <a href="https://aka.ms/talangs">this</a> supported languages in Text Analytics API.
*
* <p><strong>Code Sample</strong></p>
* {@codesnippet com.azure.ai.textanalytics.TextAnalyticsClient.beginAnalyze#Iterable-String-JobManifestTasks-AnalyzeTasksOptions-Context}
*
* @param documents A list of {@link TextDocumentInput documents} to be analyzed.
* @param displayName Job's display name.
* @param jobManifestTasks A collection of analyze tasks, such as entity recognition, PII, and key phrase tasks.
* @param options The additional configurable {@link AnalyzeTasksOptions options} that may be passed when
* analyzing a collection of tasks.
* @param context Additional context that is passed through the Http pipeline during the service call.
*
* @return A {@link SyncPoller} that polls the analyze a collection of tasks operation until it has completed,
* has failed, or has been cancelled. The completed operation returns a {@link PagedIterable} of
* {@link AnalyzeTasksResult}.
*
* @throws TextAnalyticsException If analyze operation fails.
mssfang marked this conversation as resolved.
Show resolved Hide resolved
* @throws NullPointerException If {@code jobId} is null.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There s no Job Id required in the parameters?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No Job ID is required. will remove this null pointer exception

*/
@ServiceMethod(returns = ReturnType.COLLECTION)
public SyncPoller<TextAnalyticsOperationResult, PagedIterable<AnalyzeTasksResult>> beginAnalyze(
Iterable<TextDocumentInput> documents, String displayName, JobManifestTasks jobManifestTasks,
AnalyzeTasksOptions options, Context context) {
return client.analyzeTasksAsyncClient.beginAnalyzeIterable(documents, displayName, jobManifestTasks, options,
context).getSyncPoller();
}

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.

package com.azure.ai.textanalytics.implementation;

import com.azure.ai.textanalytics.models.AnalyzeTasksResult;
import com.azure.ai.textanalytics.models.TextAnalyticsError;
import com.azure.ai.textanalytics.models.TextDocumentBatchStatistics;
import com.azure.ai.textanalytics.util.ExtractKeyPhrasesResultCollection;
import com.azure.ai.textanalytics.util.RecognizeEntitiesResultCollection;
import com.azure.ai.textanalytics.util.RecognizePiiEntitiesResultCollection;

import java.util.List;

/**
* The helper class to set the non-public properties of an {@link AnalyzeTasksResult} instance.
*/
public final class AnalyzeTasksResultPropertiesHelper {
private static AnalyzeTasksResultAccessor accessor;

private AnalyzeTasksResultPropertiesHelper() { }

/**
* Type defining the methods to set the non-public properties of an {@link AnalyzeTasksResult} instance.
*/
public interface AnalyzeTasksResultAccessor {
void setErrors(AnalyzeTasksResult analyzeTasksResult, List<TextAnalyticsError> errors);
void setStatistics(AnalyzeTasksResult analyzeTasksResult, TextDocumentBatchStatistics statistics);
void setCompleted(AnalyzeTasksResult analyzeTasksResult, int completed);
void setFailed(AnalyzeTasksResult analyzeTasksResult, int failed);
void setInProgress(AnalyzeTasksResult analyzeTasksResult, int inProgress);
void setTotal(AnalyzeTasksResult analyzeTasksResult, int total);
void setEntityRecognitionTasks(AnalyzeTasksResult analyzeTasksResult,
List<RecognizeEntitiesResultCollection> entityRecognitionTasks);
void setEntityRecognitionPiiTasks(AnalyzeTasksResult analyzeTasksResult,
List<RecognizePiiEntitiesResultCollection> entityRecognitionPiiTasks);
void setKeyPhraseExtractionTasks(AnalyzeTasksResult analyzeTasksResult,
List<ExtractKeyPhrasesResultCollection> keyPhraseExtractionTasks);
}

/**
* The method called from {@link AnalyzeTasksResult} to set it's accessor.
*
* @param analyzeTasksResultAccessor The accessor.
*/
public static void setAccessor(final AnalyzeTasksResultAccessor analyzeTasksResultAccessor) {
accessor = analyzeTasksResultAccessor;
}

public static void setErrors(AnalyzeTasksResult analyzeTasksResult, List<TextAnalyticsError> errors) {
accessor.setErrors(analyzeTasksResult, errors);
}

public static void setStatistics(AnalyzeTasksResult analyzeTasksResult, TextDocumentBatchStatistics statistics) {
accessor.setStatistics(analyzeTasksResult, statistics);
}

public static void setCompleted(AnalyzeTasksResult analyzeTasksResult, int completed) {
accessor.setCompleted(analyzeTasksResult, completed);
}

public static void setFailed(AnalyzeTasksResult analyzeTasksResult, int failed) {
accessor.setFailed(analyzeTasksResult, failed);
}

public static void setInProgress(AnalyzeTasksResult analyzeTasksResult, int inProgress) {
accessor.setInProgress(analyzeTasksResult, inProgress);
}

public static void setTotal(AnalyzeTasksResult analyzeTasksResult, int total) {
accessor.setTotal(analyzeTasksResult, total);
}

public static void setEntityRecognitionTasks(AnalyzeTasksResult analyzeTasksResult,
List<RecognizeEntitiesResultCollection> entityRecognitionTasks) {
accessor.setEntityRecognitionTasks(analyzeTasksResult, entityRecognitionTasks);
}

public static void setEntityRecognitionPiiTasks(AnalyzeTasksResult analyzeTasksResult,
List<RecognizePiiEntitiesResultCollection> entityRecognitionPiiTasks) {
accessor.setEntityRecognitionPiiTasks(analyzeTasksResult, entityRecognitionPiiTasks);
}

public static void setKeyPhraseExtractionTasks(AnalyzeTasksResult analyzeTasksResult,
List<ExtractKeyPhrasesResultCollection> keyPhraseExtractionTasks) {
accessor.setKeyPhraseExtractionTasks(analyzeTasksResult, keyPhraseExtractionTasks);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ private HealthcareTaskResultPropertiesHelper() { }
public interface HealthcareTaskResultAccessor {
void setResult(HealthcareTaskResult healthcareTaskResult,
RecognizeHealthcareEntitiesResultCollection recognizeHealthcareEntitiesResultCollection);
void setError(HealthcareTaskResult healthcareTaskResult, List<TextAnalyticsError> errors);
void setErrors(HealthcareTaskResult healthcareTaskResult, List<TextAnalyticsError> errors);
}

/**
Expand All @@ -42,6 +42,6 @@ public static void setResult(HealthcareTaskResult healthcareTaskResult,
}

public static void setErrors(HealthcareTaskResult healthcareTaskResult, List<TextAnalyticsError> errors) {
accessor.setError(healthcareTaskResult, errors);
accessor.setErrors(healthcareTaskResult, errors);
}
}
Loading