Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TA] Add analyze tasks feature support #17267

Merged
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
7b0e24c
TA-Healthcare
mssfang Nov 5, 2020
446bb42
remove checkstyle supppresion
mssfang Nov 5, 2020
5ac3e30
regenerate swagger ater a new change merged in
mssfang Nov 5, 2020
c03aa75
transfer to laptop
mssfang Nov 5, 2020
968acb9
cancellation is working now
mssfang Nov 6, 2020
1b5dbc8
address mari's most feedbacks
mssfang Nov 6, 2020
53e0d0e
replace a wrong json file for async pagination test
mssfang Nov 6, 2020
53654d6
init analyze tasks
mssfang Nov 6, 2020
8d79350
removed xxxTaskState class and TaskStete
mssfang Nov 6, 2020
5c7f4ce
add changelog
mssfang Nov 6, 2020
8c57c6c
resolve conflict and updates changes
mssfang Nov 6, 2020
37c3bf1
resolved conflict and add more tests
mssfang Nov 11, 2020
febe0be
regenerate code base on swagger 5ef5a597b3f2342bfd254ed79b97b2fe160e50a1
mssfang Nov 13, 2020
9a25060
address feedbacks
mssfang Nov 13, 2020
ee4ae09
improve PLAYBACK test speed
mssfang Nov 13, 2020
cd58ae8
checekstyle issue
mssfang Nov 13, 2020
d1c02b5
update Analyze API endpoint
mssfang Nov 17, 2020
c94420c
add issue link to TODO lists
mssfang Nov 17, 2020
8802740
renaming some APIs and address feedbacks
mssfang Nov 18, 2020
ae88549
make final class if possible
mssfang Nov 18, 2020
047f2b5
make HealthcareEntityCollection final class
mssfang Nov 18, 2020
201e025
service already fixed the nextLink to @nextLink
mssfang Nov 18, 2020
d52decd
add NPE and add live test setup
mssfang Nov 19, 2020
668650b
address last feedbacks
mssfang Nov 19, 2020
b819e4f
update readme content and links
mssfang Nov 19, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion sdk/textanalytics/azure-ai-textanalytics/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
# Release History
## 5.1.0-beta.3 (Unreleased)
**New features**
samvaity marked this conversation as resolved.
Show resolved Hide resolved
- Added support for Healthcare analysis, it is a long-running operation, and the cancellation supported.
- Added support for healthcare analysis feature. It is represented as a long-running operation. Cancellation supported.
- Added support for analyze tasks feature, It analyzes multiple tasks (such as, entity recognition, PII entity recognition
and key phrases extraction) simultaneously in a list of document.

## 5.1.0-beta.2 (2020-10-06)
### Breaking changes
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
import com.azure.core.util.polling.PollingContext;
import reactor.core.publisher.Mono;

import java.time.Duration;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
Expand All @@ -44,7 +45,7 @@
import java.util.stream.Collectors;

import static com.azure.ai.textanalytics.TextAnalyticsAsyncClient.COGNITIVE_TRACING_NAMESPACE_VALUE;
import static com.azure.ai.textanalytics.implementation.Utility.DEFAULT_POLL_DURATION;
import static com.azure.ai.textanalytics.implementation.Utility.DEFAULT_POLL_INTERVAL;
import static com.azure.ai.textanalytics.implementation.Utility.inputDocumentsValidation;
import static com.azure.ai.textanalytics.implementation.Utility.parseModelId;
import static com.azure.ai.textanalytics.implementation.Utility.parseNextLink;
Expand Down Expand Up @@ -74,12 +75,17 @@ PollerFlux<TextAnalyticsOperationResult, PagedFlux<HealthcareTaskResult>> beginA
try {
inputDocumentsValidation(documents);
String modelVersion = null;
Duration pollInterval = DEFAULT_POLL_INTERVAL;
if (options != null) {
modelVersion = options.getModelVersion();
pollInterval = options.getPollInterval();
samvaity marked this conversation as resolved.
Show resolved Hide resolved
}
// the variable used in the lambda function has to be 'final' or 'effective final'.
samvaity marked this conversation as resolved.
Show resolved Hide resolved
final Boolean finalIncludeStatistics = options == null ? null : options.isIncludeStatistics();
final Integer finalTop = options == null ? null : options.getTop();
final Integer finalSkip = options == null ? null : options.getSkip();
return new PollerFlux<>(
DEFAULT_POLL_DURATION,
pollInterval,
activationOperation(service.healthWithResponseAsync(
new MultiLanguageBatchInput().setDocuments(toMultiLanguageInput(documents)),
modelVersion,
Expand All @@ -92,12 +98,12 @@ PollerFlux<TextAnalyticsOperationResult, PagedFlux<HealthcareTaskResult>> beginA
parseModelId(healthResponse.getDeserializedHeaders().getOperationLocation()));
return textAnalyticsOperationResult;
})),
pollingOperation(jobId -> service.healthStatusWithResponseAsync(jobId, null, null,
finalIncludeStatistics, context)),
pollingOperation(jobId -> service.healthStatusWithResponseAsync(jobId,
finalTop, finalSkip, finalIncludeStatistics, context)),
(activationResponse, pollingContext) ->
monoError(logger, new RuntimeException("Use the `beginCancelHealthcareJob` to cancel the job")),
fetchingOperation(resultId -> Mono.just(getHealthcareFluxPage(resultId,
finalIncludeStatistics == null ? false : finalIncludeStatistics, context)))
fetchingOperation(resultId -> Mono.just(getHealthcareFluxPage(resultId, finalTop, finalSkip,
finalIncludeStatistics, context)))
);
} catch (RuntimeException ex) {
return PollerFlux.error(ex);
Expand All @@ -109,12 +115,16 @@ PollerFlux<TextAnalyticsOperationResult, PagedIterable<HealthcareTaskResult>> be
try {
inputDocumentsValidation(documents);
String modelVersion = null;
Duration pollInterval = DEFAULT_POLL_INTERVAL;
if (options != null) {
modelVersion = options.getModelVersion();
pollInterval = options.getPollInterval();
}
final Boolean finalIncludeStatistics = options == null ? null : options.isIncludeStatistics();
final Integer finalTop = options == null ? null : options.getTop();
final Integer finalSkip = options == null ? null : options.getSkip();
return new PollerFlux<>(
DEFAULT_POLL_DURATION,
pollInterval,
mssfang marked this conversation as resolved.
Show resolved Hide resolved
activationOperation(service.healthWithResponseAsync(
new MultiLanguageBatchInput().setDocuments(toMultiLanguageInput(documents)),
modelVersion,
Expand All @@ -132,21 +142,22 @@ PollerFlux<TextAnalyticsOperationResult, PagedIterable<HealthcareTaskResult>> be
(activationResponse, pollingContext) ->
monoError(logger, new RuntimeException("Use the `beginCancelHealthcareJob` to cancel the job")),
fetchingOperationIterable(resultId -> Mono.just(new PagedIterable<>(getHealthcareFluxPage(resultId,
finalIncludeStatistics == null ? false : finalIncludeStatistics, context))))
finalTop, finalSkip, finalIncludeStatistics, context))))
);
} catch (RuntimeException ex) {
return PollerFlux.error(ex);
}
}

PagedFlux<HealthcareTaskResult> getHealthcareFluxPage(UUID jobID, boolean showStats, Context context) {
PagedFlux<HealthcareTaskResult> getHealthcareFluxPage(UUID jobID, Integer top, Integer skip, Boolean showStats,
Context context) {
return new PagedFlux<>(
() -> getPage(null, jobID, showStats, context),
continuationToken -> getPage(continuationToken, jobID, showStats, context));
() -> getPage(null, jobID, top, skip, showStats, context),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seeing the same jobID here too - #17234 (comment)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leave the comment over there.

continuationToken -> getPage(continuationToken, jobID, top, skip, showStats, context));
}

Mono<PagedResponse<HealthcareTaskResult>> getPage(String continuationToken, UUID jobID,
boolean showStats, Context context) {
Mono<PagedResponse<HealthcareTaskResult>> getPage(String continuationToken, UUID jobID, Integer top, Integer skip,
Boolean showStats, Context context) {
samvaity marked this conversation as resolved.
Show resolved Hide resolved
try {
if (continuationToken != null) {
final Map<String, Integer> continuationTokenMap = parseNextLink(continuationToken);
Expand All @@ -156,7 +167,7 @@ Mono<PagedResponse<HealthcareTaskResult>> getPage(String continuationToken, UUID
.map(this::toTextAnalyticsPagedResponse)
.onErrorMap(Utility::mapToHttpResponseExceptionIfExist);
} else {
return service.healthStatusWithResponseAsync(jobID, null, null, showStats, context)
return service.healthStatusWithResponseAsync(jobID, top, skip, showStats, context)
.map(this::toTextAnalyticsPagedResponse)
.onErrorMap(Utility::mapToHttpResponseExceptionIfExist);
}
Expand All @@ -174,8 +185,8 @@ private PagedResponse<HealthcareTaskResult> toTextAnalyticsPagedResponse(
final List<TextAnalyticsError> errors = healthcareJobState.getErrors();

final HealthcareTaskResult healthcareTaskResult = new HealthcareTaskResult(
// TODO: change back to UUID after service support it.
healthcareJobState.getJobId().toString(),
// TODO: [Service-Bug] change back to UUID after service support it.
healthcareJobState.getJobId(),
healthcareJobState.getCreatedDateTime(),
healthcareJobState.getLastUpdateDateTime(),
toJobState(healthcareJobState.getStatus()),
Expand All @@ -196,12 +207,13 @@ private PagedResponse<HealthcareTaskResult> toTextAnalyticsPagedResponse(
null);
}

PollerFlux<TextAnalyticsOperationResult, Void> beginCancelAnalyzeHealthcare(UUID jobId, Context context) {
PollerFlux<TextAnalyticsOperationResult, Void> beginCancelAnalyzeHealthcare(String healthTaskId,
RecognizeHealthcareEntityOptions options, Context context) {
try {
Objects.requireNonNull(jobId, "'jobId' is required and cannot be null.");
Objects.requireNonNull(healthTaskId, "'healthTaskId' is required and cannot be null.");
return new PollerFlux<>(
DEFAULT_POLL_DURATION,
activationOperation(service.cancelHealthJobWithResponseAsync(jobId,
options == null ? DEFAULT_POLL_INTERVAL : options.getPollInterval(),
activationOperation(service.cancelHealthJobWithResponseAsync(UUID.fromString(healthTaskId),
samvaity marked this conversation as resolved.
Show resolved Hide resolved
context.addData(AZ_TRACING_NAMESPACE_KEY, COGNITIVE_TRACING_NAMESPACE_VALUE))
.map(healthResponse -> {
final TextAnalyticsOperationResult textAnalyticsOperationResult =
Expand Down Expand Up @@ -286,7 +298,7 @@ private Mono<PollResponse<TextAnalyticsOperationResult>> processAnalyzeModelResp
logger.info("LongRunningOperation-Cancelling");
status = LongRunningOperationStatus.IN_PROGRESS;
break;
case NOTSTARTED:
case NOT_STARTED:
case RUNNING:
status = LongRunningOperationStatus.IN_PROGRESS;
break;
Expand Down
Loading