Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[text analytics] update samples #10100

Merged
merged 14 commits into from
Mar 6, 2020
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 45 additions & 14 deletions sdk/textanalytics/azure-ai-textanalytics/samples/README.md
Original file line number Diff line number Diff line change
@@ -1,31 +1,34 @@
---
topic: sample
page_type: sample
languages:
- python
products:
- azure
- azure-ai-textanalytics
urlFragment: cognitiveservices-textanalytics-samples
iscai-msft marked this conversation as resolved.
Show resolved Hide resolved
---

# Samples for Azure Text Analytics client library for Python

These code samples show common scenario operations with the Azure Text Analytics client library.
The async versions of the samples (the python sample files appended with `_async`) show asynchronous operations
with Text Analytics and require Python 3.5 or later.
The async versions of the samples require [Python 3.5][python_35] or later.
iscai-msft marked this conversation as resolved.
Show resolved Hide resolved

Authenticate the client with a Cognitive Services/Text Analytics subscription key or a token credential from [azure-identity](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/identity/azure-identity):
* [sample_authentication.py](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/sample_authentication.py) ([async version](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/async_samples/sample_authentication_async.py))
You can authenticate your client with a Cognitive Services/Text Analytics API key or through Azure Active Directory with a token credential from [azure-identity][azure_identity]:
iscai-msft marked this conversation as resolved.
Show resolved Hide resolved
* See [sample_authentication.py][sample_authentication] and [sample_authentication_async.py][sample_authentication_async] for how to authenticate in the above cases
iscai-msft marked this conversation as resolved.
Show resolved Hide resolved

In a batch of documents:
* Detect language: [sample_detect_language.py](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/sample_detect_language.py) ([async version](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/async_samples/sample_detect_language_async.py))
* Recognize entities: [sample_recognize_entities.py](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/sample_recognize_entities.py) ([async version](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/async_samples/sample_recognize_entities_async.py))
* Recognize linked entities: [sample_recognize_linked_entities.py](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/sample_recognize_linked_entities.py) ([async version](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/async_samples/sample_recognize_linked_entities_async.py))
* Recognize personally identifiable information: [sample_recognize_pii_entities.py](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/sample_recognize_pii_entities.py) ([async version](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/async_samples/sample_recognize_pii_entities_async.py))
* Extract key phrases: [sample_extract_key_phrases.py](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/sample_extract_key_phrases.py) ([async version](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/async_samples/sample_extract_key_phrases_async.py))
* Analyze sentiment: [sample_analyze_sentiment.py](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/sample_analyze_sentiment.py) ([async version](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/async_samples/sample_analyze_sentiment_async.py))
These sample programs show common scenarios for the Text Analytics client's offerings.

|**File Name**|**Description**|
|----------------|-------------|
|[sample_detect_language.py][detect_language] and [sample_detect_language_async.py][detect_language_async]|Detect language in documents|
iscai-msft marked this conversation as resolved.
Show resolved Hide resolved
|[sample_recognize_entities.py][recognize_entities] and [sample_recognize_entities_async.py][recognize_entities_async]|Recognize named entities in documents|
|[sample_recognize_linked_entities.py][recognize_linked_entities] and [sample_recognize_linked_entities_async.py][recognize_linked_entities_async]|Recognize linked entities|
|[sample_recognize_pii_entities.py][recognize_pii_entities] and [sample_recognize_pii_entities_async.py][recognize_pii_entities_async]|Recognize personally identifiable information|
|[sample_extract_key_phrases.py][extract_key_phrases] and [sample_extract_key_phrases_async.py][extract_key_phrases_async]|Extract key phrases from documents|
|[sample_analyze_sentiment.py][analyze_sentiment] and [sample_analyze_sentiment_async.py][analyze_sentiment_async]|Analye the sentiment of text|
iscai-msft marked this conversation as resolved.
Show resolved Hide resolved

## Prerequisites
* Python 2.7, or 3.5 or later is required to use this package (3.5 or later if using asyncio)
* Python [2.7][python_27], or [3.5][python_35] or later is required to use this package (3.5 or later if using asyncio)
iscai-msft marked this conversation as resolved.
Show resolved Hide resolved
* You must have an [Azure subscription](https://azure.microsoft.com/free/) and an
iscai-msft marked this conversation as resolved.
Show resolved Hide resolved
[Azure Text Analytics account](https://docs.microsoft.com/azure/cognitive-services/cognitive-services-apis-create-account?tabs=singleservice%2Cwindows) to run these samples.

Expand All @@ -36,6 +39,10 @@ In a batch of documents:
```bash
pip install azure-ai-textanalytics --pre
```
* If authenticating with Azure Active Directory, make sure you have [azure-identity](https://pypi.org/project/azure-identity/) installed:
```bash
pip install azure-identity
```

2. Clone or download this sample repository
3. Open the sample folder in Visual Studio Code or your IDE of choice.
Expand All @@ -48,5 +55,29 @@ pip install azure-ai-textanalytics --pre

## Next steps

Check out the [API reference documentation](https://westus.dev.cognitive.microsoft.com/docs/services/TextAnalytics-v3-0-Preview-1/operations/Languages) to learn more about
Check out the [API reference documentation](https://aka.ms/azsdk-python-textanalytics-ref-docs) to learn more about
what you can do with the Azure Text Analytics client library.

|**Advanced Sample File Name**|**Description**|
iscai-msft marked this conversation as resolved.
Show resolved Hide resolved
|----------------|-------------|
|[sample_get_detailed_diagnostics_information.py][get_detailed_diagnostics_information] and [sample_get_detailed_diagnostics_information_async.py][get_detailed_diagnostics_information_async]|Get the request batch statistics, model version, and raw response through a callback|

[python_35]: https://www.python.org/downloads/release/python-350/
[python_27]: https://www.python.org/downloads/release/python-270/
[azure-identity]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/identity/azure-identity
[sample_authentication]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/sample_authentication.py
[sample_authentication_async]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/async_samples/sample_authentication_async.py
[detect_language]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/sample_detect_language.py
[detect_language_async]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/async_samples/sample_detect_language_async.py
[recognize_entities]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/sample_recognize_entities.py
[recognize_entities_async]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/async_samples/sample_recognize_entities_async.py
[recognize_linked_entities]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/sample_recognize_linked_entities.py
[recognize_linked_entities_async]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/async_samples/sample_recognize_linked_entities_async.py
[recognize_pii_entities]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/sample_recognize_pii_entities.py
[recognize_pii_entities_async]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/async_samples/sample_recognize_pii_entities_async.py
[extract_key_phrases]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/sample_extract_key_phrases.py
[extract_key_phrases_async]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/async_samples/sample_extract_key_phrases_async.py
[analyze_sentiment]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/sample_analyze_sentiment.py
[analyze_sentiment_async]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/async_samples/sample_analyze_sentiment_async.py
[get_detailed_diagnostics_information]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/sample_get_detailed_diagnostics_information.py
[get_detailed_diagnostics_information_async]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/textanalytics/azure-ai-textanalytics/samples/async_samples/sample_get_detailed_diagnostics_information_async.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
FILE: sample_analyze_sentiment_async.py

DESCRIPTION:
This sample demonstrates how to analyze sentiment in a batch of documents.
This sample demonstrates how to analyze sentiment in documents.
An overall and per-sentence sentiment is returned.

USAGE:
Expand Down Expand Up @@ -51,63 +51,24 @@ async def analyze_sentiment_async(self):
print("Document text: {}".format(documents[idx]))
print("Overall sentiment: {}".format(doc.sentiment))
# [END batch_analyze_sentiment_async]
print("Overall confidence scores: positive={0:.3f}; neutral={1:.3f}; negative={2:.3f} \n".format(
print("Overall confidence scores: positive={}; neutral={}; negative={} \n".format(
doc.confidence_scores.positive,
doc.confidence_scores.neutral,
doc.confidence_scores.negative,
))
for idx, sentence in enumerate(doc.sentences):
print("Sentence {} sentiment: {}".format(idx+1, sentence.sentiment))
print("Sentence confidence scores: positive={0:.3f}; neutral={1:.3f}; negative={2:.3f}".format(
print("Sentence confidence scores: positive={}; neutral={}; negative={}".format(
sentence.confidence_scores.positive,
sentence.confidence_scores.neutral,
sentence.confidence_scores.negative,
))
print("Offset: {}".format(sentence.grapheme_offset))
print("Length: {}\n".format(sentence.grapheme_length))
print("------------------------------------")

async def alternative_scenario_analyze_sentiment_async(self):
"""This sample demonstrates how to retrieve batch statistics, the
model version used, and the raw response returned from the service.

It additionally shows an alternative way to pass in the input documents
using a list[TextDocumentInput] and supplying your own IDs and language hints along
with the text.
"""
from azure.ai.textanalytics.aio import TextAnalyticsClient
from azure.ai.textanalytics import TextAnalyticsApiKeyCredential
text_analytics_client = TextAnalyticsClient(endpoint=self.endpoint, credential=TextAnalyticsApiKeyCredential(self.key))

documents = [
{"id": "0", "language": "en", "text": "I had the best day of my life."},
{"id": "1", "language": "en",
"text": "This was a waste of my time. The speaker put me to sleep."},
{"id": "2", "language": "es", "text": "No tengo dinero ni nada que dar..."},
{"id": "3", "language": "fr",
"text": "L'hôtel n'était pas très confortable. L'éclairage était trop sombre."}
]

extras = []

def callback(resp):
extras.append(resp.statistics)
extras.append(resp.model_version)
extras.append(resp.raw_response)

async with text_analytics_client:
result = await text_analytics_client.analyze_sentiment(
documents,
show_stats=True,
model_version="latest",
raw_response_hook=callback
)


async def main():
sample = AnalyzeSentimentSampleAsync()
await sample.analyze_sentiment_async()
await sample.alternative_scenario_analyze_sentiment_async()


if __name__ == '__main__':
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,11 @@
FILE: sample_authentication_async.py

DESCRIPTION:
This sample demonstrates how to authenticate with the text analytics service.
This sample demonstrates how to authenticate to the text analytics service.
iscai-msft marked this conversation as resolved.
Show resolved Hide resolved

There are two supported methods of authentication:
1) Use a cognitive services/text analytics API key with TextAnalyticsApiKeyCredential
iscai-msft marked this conversation as resolved.
Show resolved Hide resolved
2) Use a token credential to authenticate with Azure Active Directory
2) Use a token credential from azure-identity to authenticate with Azure Active Directory

See more details about authentication here:
https://docs.microsoft.com/azure/cognitive-services/authentication
Expand All @@ -37,6 +37,7 @@
class AuthenticationSampleAsync(object):

async def authentication_with_api_key_credential_async(self):
print("\n.. authentication_with_api_key_credential_async")
# [START create_ta_client_with_key_async]
from azure.ai.textanalytics.aio import TextAnalyticsClient
from azure.ai.textanalytics import TextAnalyticsApiKeyCredential
Expand All @@ -54,9 +55,10 @@ async def authentication_with_api_key_credential_async(self):
print("Confidence score: {}".format(result[0].primary_language.score))

async def authentication_with_azure_active_directory_async(self):
"""DefaultAzureCredential will use the values from the environment
"""DefaultAzureCredential will use the values from these environment
variables: AZURE_CLIENT_ID, AZURE_TENANT_ID, AZURE_CLIENT_SECRET
"""
print("\n.. authentication_with_azure_active_directory_async")
# [START create_ta_client_with_aad_async]
from azure.ai.textanalytics.aio import TextAnalyticsClient
from azure.identity.aio import DefaultAzureCredential
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -55,46 +55,10 @@ async def detect_language_async(self):
print(doc.id, doc.error)
# [END batch_detect_language_async]

async def alternative_scenario_detect_language_async(self):
"""This sample demonstrates how to retrieve batch statistics, the
model version used, and the raw response returned from the service.

It additionally shows an alternative way to pass in the input documents
using a list[DetectLanguageInput] and supplying your own IDs and country hints along
with the text.
"""
from azure.ai.textanalytics.aio import TextAnalyticsClient
from azure.ai.textanalytics import TextAnalyticsApiKeyCredential
text_analytics_client = TextAnalyticsClient(endpoint=self.endpoint, credential=TextAnalyticsApiKeyCredential(self.key))

documents = [
{"id": "0", "country_hint": "US", "text": "This is a document written in English."},
{"id": "1", "country_hint": "MX", "text": "Este es un document escrito en Español."},
{"id": "2", "country_hint": "CN", "text": "这是一个用中文写的文件"},
{"id": "3", "country_hint": "DE", "text": "Dies ist ein Dokument in englischer Sprache."},
{"id": "4", "country_hint": "SE", "text": "Detta är ett dokument skrivet på engelska."}
]

extras = []

def callback(resp):
extras.append(resp.statistics)
extras.append(resp.model_version)
extras.append(resp.raw_response)

async with text_analytics_client:
result = await text_analytics_client.detect_language(
documents,
show_stats=True,
model_version="latest",
raw_response_hook=callback
)


async def main():
sample = DetectLanguageSampleAsync()
await sample.detect_language_async()
await sample.alternative_scenario_detect_language_async()


if __name__ == '__main__':
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,45 +50,10 @@ async def extract_key_phrases_async(self):
print(doc.id, doc.error)
# [END batch_extract_key_phrases_async]

async def alternative_scenario_extract_key_phrases_async(self):
"""This sample demonstrates how to retrieve batch statistics, the
model version used, and the raw response returned from the service.

It additionally shows an alternative way to pass in the input documents
using a list[TextDocumentInput] and supplying your own IDs and language hints along
with the text.
"""
from azure.ai.textanalytics.aio import TextAnalyticsClient
from azure.ai.textanalytics import TextAnalyticsApiKeyCredential
text_analytics_client = TextAnalyticsClient(endpoint=self.endpoint, credential=TextAnalyticsApiKeyCredential(self.key))

documents = [
{"id": "0", "language": "en",
"text": "Redmond is a city in King County, Washington, United States, located 15 miles east of Seattle."},
{"id": "1", "language": "en",
"text": "I need to take my cat to the veterinarian."},
{"id": "2", "language": "en", "text": "I will travel to South America in the summer."}
]
extras = []

def callback(resp):
extras.append(resp.statistics)
extras.append(resp.model_version)
extras.append(resp.raw_response)

async with text_analytics_client:
result = await text_analytics_client.extract_key_phrases(
documents,
show_stats=True,
model_version="latest",
raw_response_hook=callback
)


async def main():
sample = ExtractKeyPhrasesSampleAsync()
await sample.extract_key_phrases_async()
await sample.alternative_scenario_extract_key_phrases_async()


if __name__ == '__main__':
Expand Down
Loading