feature: support local mode in SageMaker Studio (#1300) #4347
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* feature: support local mode in SageMaker Studio * chore: fix typo * chore: fix formatting * chore: revert changes for docker compose logs * chore: black-format * change: Use predtermined dns-allow-listed-hostname for Studio Local Support * add support for CodeEditor and JupyterLabs --------- Co-authored-by: Mufaddal Rohawala <mufi@amazon.com> Co-authored-by: Mufaddal Rohawala <89424143+mufaddal-rohawala@users.noreply.github.com>
mufaddal-rohawala
requested review from
benieric
and removed request for
a team
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## master #4347 +/- ##
==========================================
- Coverage 86.88% 86.78% -0.11%
==========================================
Files 1197 380 -817
Lines 106446 35107 -71339
==========================================
- Hits 92488 30468 -62020
+ Misses 13958 4639 -9319 ☔ View full report in Codecov by Sentry. |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
|
Have verified Unit test: tests/unit/sagemaker/workflow/test_transform_step.py::test_transform_step_with_transformer VERIFIED passes locally. Following PR Test failures are affecting all PR checks and are unrelated to the change: Slow test - FAILED tests/integ/test_inference_recommender.py::test_deploy_inference_recommendation_id_with_registered_model_sklearn |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
akrishna1995
approved these changes
Dec 28, 2023
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
suryans-commit
pushed a commit
to suryans-commit/sagemaker-python-sdk
that referenced
this pull request
Feb 5, 2024
* feature: support local mode in SageMaker Studio * chore: fix typo * chore: fix formatting * chore: revert changes for docker compose logs * chore: black-format * change: Use predtermined dns-allow-listed-hostname for Studio Local Support * add support for CodeEditor and JupyterLabs --------- Co-authored-by: Erick Benitez-Ramos <141277478+benieric@users.noreply.github.com> Co-authored-by: Mufaddal Rohawala <mufi@amazon.com>
benieric
added a commit
that referenced
this pull request
Feb 8, 2024
…ngestion. (#4413) * change: update image_uri_configs 12-13-2023 12:23:06 PST * change: update image_uri_configs 12-13-2023 14:04:54 PST * prepare release v2.200.1 * update development version to v2.200.2.dev0 * fix: Move func and args serialization of function step to step level (#4312) * fix: Add write permission to job output dirs for remote and step decorator running on non-root job user (#4325) * feat: Added update for model package (#4309) Co-authored-by: Keshav Chandak <chakesh@amazon.com> * documentation: fix ModelBuilder sample notebook links (#4319) * feat: Use specific images for SMP v2 jobs (#4333) * Add check for smp lib * update syntax * Remove unused images * Update repo name and regions * Update account number * Update framework name and check for None distribution * Add unit tests for smp v2 uri * Check enabled * Remove logging * Add cuda version in uri * Update cu121 * Update syntax * Fix black check * Fix black --------- Co-authored-by: huilgolr <yoda@ip-10-0-12-252.us-west-2.compute.internal> * Fix: Updated js mb compression logic - ModelBuilder (#4294) Co-authored-by: EC2 Default User <ec2-user@ip-172-16-54-104.us-west-2.compute.internal> * documentation: SMP v2 doc updates (#1423) (#4336) * doc update for estimator distribution art * add note to the SMP doc and minor fixes * remove subnodes * rm all v1 content as documenting everything in aws docs * fix build errors * fix white spaces * rm smdistributed from TF estimator distribution * rm white spaces * add notes to TF estimator distribution * fix links * incorporate feedback * update example values * fix version numbers in the notes Co-authored-by: Miyoung <cmiyoung@amazon.com> * prepare release v2.201.0 * update development version to v2.201.1.dev0 * Fix: Add additional model builder telemetry (#4334) * move telemetry code to public * add additional test --------- Co-authored-by: EC2 Default User <ec2-user@ip-172-16-54-104.us-west-2.compute.internal> * feature: support remote debug for sagemaker training job (#4315) * feature: support remote debug for sagemaker training job * change: Replace update_remote_config with 2 helper methods for enable and disable respectively * change: add new argument enable_remote_debug to skip set of test_jumpstart_estimator_kwargs_match_parent_class * chore: add jumpstart support for remote debug --------- Co-authored-by: Xinyu Xie <xixinyu@amazon.com> Co-authored-by: Evan Kravitz <evakravi@amazon.com> * Update tblib constraint (#4317) * Fix: Fix job_objective type (#4303) * change: update image_uri_configs 12-21-2023 08:32:41 PST * prepare release v2.202.0 * update development version to v2.202.1.dev0 * Using logging instead of prints (#4133) * documentation: update issue template. (#4337) * change: update model path in local mode (#4296) * Update model path in local mode * Add test * change: update image_uri_configs 12-22-2023 06:17:35 PST * prepare release v2.202.1 * update development version to v2.202.2.dev0 * change: create role if needed in `get_execution_role` (#4323) * Create role if needed in get_execution_role * Add tests * Change: More pythonic tags (#4327) * Change: More pythonic tags * Fix broken tags * More tags formatting and add a test * Fix tests * Raise Exception for debug (#4344) Co-authored-by: Ruilian Gao <ruiliann@amazon.com> * Change: Allow extra_args to be passed to uploader (#4338) * Change: Allow extra_args to be passed to uploader * Fix tests * Black * Fix test * Change: Drop py2 tag from the wheel as we don't support Python 2 (#4343) * Disable failed test in IR (#4345) * Disable failed test in IR * Fix format --------- Co-authored-by: Ruilian Gao <ruiliann@amazon.com> * change: update image_uri_configs 12-25-2023 06:17:33 PST * feat: Supporting tbac in load_run (#4039) * feature: support local mode in SageMaker Studio (#1300) (#4347) * feature: support local mode in SageMaker Studio * chore: fix typo * chore: fix formatting * chore: revert changes for docker compose logs * chore: black-format * change: Use predtermined dns-allow-listed-hostname for Studio Local Support * add support for CodeEditor and JupyterLabs --------- Co-authored-by: Erick Benitez-Ramos <141277478+benieric@users.noreply.github.com> Co-authored-by: Mufaddal Rohawala <mufi@amazon.com> * prepare release v2.203.0 * update development version to v2.203.1.dev0 * change: update image_uri_configs 12-29-2023 06:17:34 PST * query hf api for model md (#4346) Co-authored-by: EC2 Default User <ec2-user@ip-172-16-54-104.us-west-2.compute.internal> * fix: skip failing integs (#4348) Co-authored-by: Mufaddal Rohawala <mufi@amazon.com> * change: TGI 1.3.3 (#4335) * prepare release v2.203.1 * update development version to v2.203.2.dev0 * feat: parallelize notebook search utils, add new operators (#4342) * feat: parallelize notebook search utils * chore: raise exception in notebook utils if thread has error * chore: improve variable name * fix: not passing region to get jumpstart bucket * chore: add sagemaker session to notebook utils * chore: address PR comments * feat: add support for includes, begins with, ends with * fix: pylint * feat: private util for model eula key * fix: unit tests, use verify_model_region_and_return_specs in notebook utils * Revert "feat: private util for model eula key" This reverts commit e2daefc. * chore: add search keywords to header * fix: change ConditionNot incorrect property Expression to Condition (#4351) * fix: Huggingface glue failing tests (#4367) * fix: Huggingface glue failing tests * fix: Sphinx doc build failure * fix: Huggingface glue failing tests * fix: failing sphinx tests * fix: failing sphinx tests * fix: failing black check * fix: sphinx doc errors * fix: sphinx doc errors * sphinx * black-format * sphinx * sphinx * sphinx --------- Co-authored-by: Mufaddal Rohawala <mufi@amazon.com> Co-authored-by: Erick Benitez-Ramos <benieric@amazon.com> * fix: Add PyTorch 2.1.0 SM Training DLC to UNSUPPORTED_DLC_IMAGE_FOR_SM_PARALLELISM list (#4356) * add 2.1 unsupported smddp * formatting * feat: Support custom repack model settings (#4328) * change: update sphinx version (#4377) * change: update sphinx version * Update sphinx * change: Updates for DJL 0.26.0 release (#4366) * change: TGI NeuronX (#4375) * TGI NeuronX * Update * Update * fix: add warning message for job-prefixed pipeline steps when no job name is provided (#4371) Co-authored-by: svia3 <svia@amazon.com> * change: JumpStart - TLV region launch (#4379) * feat: add throughput management support for feature group (#4359) * feat: add throughput management support for feature group * documentation: add doc for feature group throughput config --------- Co-authored-by: Nilesh PS <psnilesh@amazon.com> * change: Enable galactus integ tests (#4376) * feat: Enable galactus integ tests * fix flake8 * fix doc8 * trying to see if it works with slow tests * small fixes in import error * fix missing import * try to remove some dependencies from requirement to see if pr test can be fixed * fix flake8 * Enable more tests * Add rerun annotation and further remove dependencies * comment out 2 integ tests * Remove local mode test for now * fix flake8 * prepare release v2.204.0 * update development version to v2.204.1.dev0 * fix: Add validation for empty ParameterString value in start local pipeline (#4354) * feat: Support selective pipeline execution for function step (#4372) * change: update image_uri_configs 01-24-2024 06:17:33 PST * fix: update get_execution_role_arn from metadata file if present (#4388) * fix: Support using PipelineDefinitionConfig in local mode (#4352) * fix: remove fastapi and uvicorn dependencies (#4365) They are not used in the codebase. Closes #4361 #4295 * prepare release v2.205.0 * update development version to v2.205.1.dev0 * change: TGI NeuronX 0.0.17 (#4390) * fix: Support PipelineVariable for ModelQualityCheckConfig attributes (#4353) * feat: Logic to detect hardware GPU count and aggregate GPU memory size in MiB (#4389) * Add logic to detect hardware GPU count and aggregate GPU memory size in MiB * Fix all formatting * Addressed PR review comments * Addressed PR Review messages * Addressed PR Review Messages * Addressed PR Review comments * Addressed PR Review Comments * Add integration tests * Add config * Fix integration tests * Include Instance Types GPU infor Config files * Addressed PR review comments * Fix unit tests * Fix unit test: 'Mock' object is not subscriptable --------- Co-authored-by: Jonathan Makunga <makung@amazon.com> * fix: fixed create monitoring schedule failing after validation error (#4385) Co-authored-by: Keshav Chandak <chakesh@amazon.com> * Add collection type support for Feaure Group Ingestion. Add TargetStores support for PutRecord and Ingestion. * Remove merge conflicts. * Update the feature definition type * Black formatting * Fix Flake8 formatting * Fix Pylint * Fix Formatting. --------- Co-authored-by: sagemaker-bot <sagemaker-bot@amazon.com> Co-authored-by: ci <ci> Co-authored-by: qidewenwhen <32910701+qidewenwhen@users.noreply.github.com> Co-authored-by: Keshav Chandak <keshav.chandak1995@gmail.com> Co-authored-by: Keshav Chandak <chakesh@amazon.com> Co-authored-by: stacicho <stacicho@amazon.com> Co-authored-by: Teng-xu <67929972+Teng-xu@users.noreply.github.com> Co-authored-by: huilgolr <yoda@ip-10-0-12-252.us-west-2.compute.internal> Co-authored-by: Gary Wang <38331932+gwang111@users.noreply.github.com> Co-authored-by: EC2 Default User <ec2-user@ip-172-16-54-104.us-west-2.compute.internal> Co-authored-by: akrishna1995 <38850354+akrishna1995@users.noreply.github.com> Co-authored-by: Miyoung <cmiyoung@amazon.com> Co-authored-by: Xinyu Xie <xiexinyucrab@126.com> Co-authored-by: Xinyu Xie <xixinyu@amazon.com> Co-authored-by: Evan Kravitz <evakravi@amazon.com> Co-authored-by: martinRenou <martin.renou@gmail.com> Co-authored-by: Duc Trung Le <leductrungxf@gmail.com> Co-authored-by: ruiliann666 <141953824+ruiliann666@users.noreply.github.com> Co-authored-by: Ruilian Gao <ruiliann@amazon.com> Co-authored-by: ananth102 <ananthbashyam1@gmail.com> Co-authored-by: Mufaddal Rohawala <89424143+mufaddal-rohawala@users.noreply.github.com> Co-authored-by: Erick Benitez-Ramos <141277478+benieric@users.noreply.github.com> Co-authored-by: Mufaddal Rohawala <mufi@amazon.com> Co-authored-by: amzn-choeric <105388439+amzn-choeric@users.noreply.github.com> Co-authored-by: evakravi <69981223+evakravi@users.noreply.github.com> Co-authored-by: Erick Benitez-Ramos <benieric@amazon.com> Co-authored-by: Sirut Buasai <73297481+sirutBuasai@users.noreply.github.com> Co-authored-by: Sindhu Somasundaram <56774226+sindhuvahinis@users.noreply.github.com> Co-authored-by: Stephen Via <51342648+svia3@users.noreply.github.com> Co-authored-by: svia3 <svia@amazon.com> Co-authored-by: Haixin Wang <98612668+haixiw@users.noreply.github.com> Co-authored-by: Nilesh PS <nps17thatsme@gmail.com> Co-authored-by: Nilesh PS <psnilesh@amazon.com> Co-authored-by: jiapinw <95885824+jiapinw@users.noreply.github.com> Co-authored-by: Jay Goyani <135654128+jgoyani1@users.noreply.github.com> Co-authored-by: Justin <justinm088@hotmail.com> Co-authored-by: Jonathan Makunga <54963715+makungaj1@users.noreply.github.com> Co-authored-by: Jonathan Makunga <makung@amazon.com>
Captainia
pushed a commit
to Captainia/sagemaker-python-sdk
that referenced
this pull request
Feb 29, 2024
…ngestion. (aws#4413) * change: update image_uri_configs 12-13-2023 12:23:06 PST * change: update image_uri_configs 12-13-2023 14:04:54 PST * prepare release v2.200.1 * update development version to v2.200.2.dev0 * fix: Move func and args serialization of function step to step level (aws#4312) * fix: Add write permission to job output dirs for remote and step decorator running on non-root job user (aws#4325) * feat: Added update for model package (aws#4309) Co-authored-by: Keshav Chandak <chakesh@amazon.com> * documentation: fix ModelBuilder sample notebook links (aws#4319) * feat: Use specific images for SMP v2 jobs (aws#4333) * Add check for smp lib * update syntax * Remove unused images * Update repo name and regions * Update account number * Update framework name and check for None distribution * Add unit tests for smp v2 uri * Check enabled * Remove logging * Add cuda version in uri * Update cu121 * Update syntax * Fix black check * Fix black --------- Co-authored-by: huilgolr <yoda@ip-10-0-12-252.us-west-2.compute.internal> * Fix: Updated js mb compression logic - ModelBuilder (aws#4294) Co-authored-by: EC2 Default User <ec2-user@ip-172-16-54-104.us-west-2.compute.internal> * documentation: SMP v2 doc updates (aws#1423) (aws#4336) * doc update for estimator distribution art * add note to the SMP doc and minor fixes * remove subnodes * rm all v1 content as documenting everything in aws docs * fix build errors * fix white spaces * rm smdistributed from TF estimator distribution * rm white spaces * add notes to TF estimator distribution * fix links * incorporate feedback * update example values * fix version numbers in the notes Co-authored-by: Miyoung <cmiyoung@amazon.com> * prepare release v2.201.0 * update development version to v2.201.1.dev0 * Fix: Add additional model builder telemetry (aws#4334) * move telemetry code to public * add additional test --------- Co-authored-by: EC2 Default User <ec2-user@ip-172-16-54-104.us-west-2.compute.internal> * feature: support remote debug for sagemaker training job (aws#4315) * feature: support remote debug for sagemaker training job * change: Replace update_remote_config with 2 helper methods for enable and disable respectively * change: add new argument enable_remote_debug to skip set of test_jumpstart_estimator_kwargs_match_parent_class * chore: add jumpstart support for remote debug --------- Co-authored-by: Xinyu Xie <xixinyu@amazon.com> Co-authored-by: Evan Kravitz <evakravi@amazon.com> * Update tblib constraint (aws#4317) * Fix: Fix job_objective type (aws#4303) * change: update image_uri_configs 12-21-2023 08:32:41 PST * prepare release v2.202.0 * update development version to v2.202.1.dev0 * Using logging instead of prints (aws#4133) * documentation: update issue template. (aws#4337) * change: update model path in local mode (aws#4296) * Update model path in local mode * Add test * change: update image_uri_configs 12-22-2023 06:17:35 PST * prepare release v2.202.1 * update development version to v2.202.2.dev0 * change: create role if needed in `get_execution_role` (aws#4323) * Create role if needed in get_execution_role * Add tests * Change: More pythonic tags (aws#4327) * Change: More pythonic tags * Fix broken tags * More tags formatting and add a test * Fix tests * Raise Exception for debug (aws#4344) Co-authored-by: Ruilian Gao <ruiliann@amazon.com> * Change: Allow extra_args to be passed to uploader (aws#4338) * Change: Allow extra_args to be passed to uploader * Fix tests * Black * Fix test * Change: Drop py2 tag from the wheel as we don't support Python 2 (aws#4343) * Disable failed test in IR (aws#4345) * Disable failed test in IR * Fix format --------- Co-authored-by: Ruilian Gao <ruiliann@amazon.com> * change: update image_uri_configs 12-25-2023 06:17:33 PST * feat: Supporting tbac in load_run (aws#4039) * feature: support local mode in SageMaker Studio (aws#1300) (aws#4347) * feature: support local mode in SageMaker Studio * chore: fix typo * chore: fix formatting * chore: revert changes for docker compose logs * chore: black-format * change: Use predtermined dns-allow-listed-hostname for Studio Local Support * add support for CodeEditor and JupyterLabs --------- Co-authored-by: Erick Benitez-Ramos <141277478+benieric@users.noreply.github.com> Co-authored-by: Mufaddal Rohawala <mufi@amazon.com> * prepare release v2.203.0 * update development version to v2.203.1.dev0 * change: update image_uri_configs 12-29-2023 06:17:34 PST * query hf api for model md (aws#4346) Co-authored-by: EC2 Default User <ec2-user@ip-172-16-54-104.us-west-2.compute.internal> * fix: skip failing integs (aws#4348) Co-authored-by: Mufaddal Rohawala <mufi@amazon.com> * change: TGI 1.3.3 (aws#4335) * prepare release v2.203.1 * update development version to v2.203.2.dev0 * feat: parallelize notebook search utils, add new operators (aws#4342) * feat: parallelize notebook search utils * chore: raise exception in notebook utils if thread has error * chore: improve variable name * fix: not passing region to get jumpstart bucket * chore: add sagemaker session to notebook utils * chore: address PR comments * feat: add support for includes, begins with, ends with * fix: pylint * feat: private util for model eula key * fix: unit tests, use verify_model_region_and_return_specs in notebook utils * Revert "feat: private util for model eula key" This reverts commit e2daefc. * chore: add search keywords to header * fix: change ConditionNot incorrect property Expression to Condition (aws#4351) * fix: Huggingface glue failing tests (aws#4367) * fix: Huggingface glue failing tests * fix: Sphinx doc build failure * fix: Huggingface glue failing tests * fix: failing sphinx tests * fix: failing sphinx tests * fix: failing black check * fix: sphinx doc errors * fix: sphinx doc errors * sphinx * black-format * sphinx * sphinx * sphinx --------- Co-authored-by: Mufaddal Rohawala <mufi@amazon.com> Co-authored-by: Erick Benitez-Ramos <benieric@amazon.com> * fix: Add PyTorch 2.1.0 SM Training DLC to UNSUPPORTED_DLC_IMAGE_FOR_SM_PARALLELISM list (aws#4356) * add 2.1 unsupported smddp * formatting * feat: Support custom repack model settings (aws#4328) * change: update sphinx version (aws#4377) * change: update sphinx version * Update sphinx * change: Updates for DJL 0.26.0 release (aws#4366) * change: TGI NeuronX (aws#4375) * TGI NeuronX * Update * Update * fix: add warning message for job-prefixed pipeline steps when no job name is provided (aws#4371) Co-authored-by: svia3 <svia@amazon.com> * change: JumpStart - TLV region launch (aws#4379) * feat: add throughput management support for feature group (aws#4359) * feat: add throughput management support for feature group * documentation: add doc for feature group throughput config --------- Co-authored-by: Nilesh PS <psnilesh@amazon.com> * change: Enable galactus integ tests (aws#4376) * feat: Enable galactus integ tests * fix flake8 * fix doc8 * trying to see if it works with slow tests * small fixes in import error * fix missing import * try to remove some dependencies from requirement to see if pr test can be fixed * fix flake8 * Enable more tests * Add rerun annotation and further remove dependencies * comment out 2 integ tests * Remove local mode test for now * fix flake8 * prepare release v2.204.0 * update development version to v2.204.1.dev0 * fix: Add validation for empty ParameterString value in start local pipeline (aws#4354) * feat: Support selective pipeline execution for function step (aws#4372) * change: update image_uri_configs 01-24-2024 06:17:33 PST * fix: update get_execution_role_arn from metadata file if present (aws#4388) * fix: Support using PipelineDefinitionConfig in local mode (aws#4352) * fix: remove fastapi and uvicorn dependencies (aws#4365) They are not used in the codebase. Closes aws#4361 aws#4295 * prepare release v2.205.0 * update development version to v2.205.1.dev0 * change: TGI NeuronX 0.0.17 (aws#4390) * fix: Support PipelineVariable for ModelQualityCheckConfig attributes (aws#4353) * feat: Logic to detect hardware GPU count and aggregate GPU memory size in MiB (aws#4389) * Add logic to detect hardware GPU count and aggregate GPU memory size in MiB * Fix all formatting * Addressed PR review comments * Addressed PR Review messages * Addressed PR Review Messages * Addressed PR Review comments * Addressed PR Review Comments * Add integration tests * Add config * Fix integration tests * Include Instance Types GPU infor Config files * Addressed PR review comments * Fix unit tests * Fix unit test: 'Mock' object is not subscriptable --------- Co-authored-by: Jonathan Makunga <makung@amazon.com> * fix: fixed create monitoring schedule failing after validation error (aws#4385) Co-authored-by: Keshav Chandak <chakesh@amazon.com> * Add collection type support for Feaure Group Ingestion. Add TargetStores support for PutRecord and Ingestion. * Remove merge conflicts. * Update the feature definition type * Black formatting * Fix Flake8 formatting * Fix Pylint * Fix Formatting. --------- Co-authored-by: sagemaker-bot <sagemaker-bot@amazon.com> Co-authored-by: ci <ci> Co-authored-by: qidewenwhen <32910701+qidewenwhen@users.noreply.github.com> Co-authored-by: Keshav Chandak <keshav.chandak1995@gmail.com> Co-authored-by: Keshav Chandak <chakesh@amazon.com> Co-authored-by: stacicho <stacicho@amazon.com> Co-authored-by: Teng-xu <67929972+Teng-xu@users.noreply.github.com> Co-authored-by: huilgolr <yoda@ip-10-0-12-252.us-west-2.compute.internal> Co-authored-by: Gary Wang <38331932+gwang111@users.noreply.github.com> Co-authored-by: EC2 Default User <ec2-user@ip-172-16-54-104.us-west-2.compute.internal> Co-authored-by: akrishna1995 <38850354+akrishna1995@users.noreply.github.com> Co-authored-by: Miyoung <cmiyoung@amazon.com> Co-authored-by: Xinyu Xie <xiexinyucrab@126.com> Co-authored-by: Xinyu Xie <xixinyu@amazon.com> Co-authored-by: Evan Kravitz <evakravi@amazon.com> Co-authored-by: martinRenou <martin.renou@gmail.com> Co-authored-by: Duc Trung Le <leductrungxf@gmail.com> Co-authored-by: ruiliann666 <141953824+ruiliann666@users.noreply.github.com> Co-authored-by: Ruilian Gao <ruiliann@amazon.com> Co-authored-by: ananth102 <ananthbashyam1@gmail.com> Co-authored-by: Mufaddal Rohawala <89424143+mufaddal-rohawala@users.noreply.github.com> Co-authored-by: Erick Benitez-Ramos <141277478+benieric@users.noreply.github.com> Co-authored-by: Mufaddal Rohawala <mufi@amazon.com> Co-authored-by: amzn-choeric <105388439+amzn-choeric@users.noreply.github.com> Co-authored-by: evakravi <69981223+evakravi@users.noreply.github.com> Co-authored-by: Erick Benitez-Ramos <benieric@amazon.com> Co-authored-by: Sirut Buasai <73297481+sirutBuasai@users.noreply.github.com> Co-authored-by: Sindhu Somasundaram <56774226+sindhuvahinis@users.noreply.github.com> Co-authored-by: Stephen Via <51342648+svia3@users.noreply.github.com> Co-authored-by: svia3 <svia@amazon.com> Co-authored-by: Haixin Wang <98612668+haixiw@users.noreply.github.com> Co-authored-by: Nilesh PS <nps17thatsme@gmail.com> Co-authored-by: Nilesh PS <psnilesh@amazon.com> Co-authored-by: jiapinw <95885824+jiapinw@users.noreply.github.com> Co-authored-by: Jay Goyani <135654128+jgoyani1@users.noreply.github.com> Co-authored-by: Justin <justinm088@hotmail.com> Co-authored-by: Jonathan Makunga <54963715+makungaj1@users.noreply.github.com> Co-authored-by: Jonathan Makunga <makung@amazon.com>
jiapinw
added a commit
to jiapinw/sagemaker-python-sdk
that referenced
this pull request
Jun 25, 2024
…ngestion. (aws#4413) * change: update image_uri_configs 12-13-2023 12:23:06 PST * change: update image_uri_configs 12-13-2023 14:04:54 PST * prepare release v2.200.1 * update development version to v2.200.2.dev0 * fix: Move func and args serialization of function step to step level (aws#4312) * fix: Add write permission to job output dirs for remote and step decorator running on non-root job user (aws#4325) * feat: Added update for model package (aws#4309) Co-authored-by: Keshav Chandak <chakesh@amazon.com> * documentation: fix ModelBuilder sample notebook links (aws#4319) * feat: Use specific images for SMP v2 jobs (aws#4333) * Add check for smp lib * update syntax * Remove unused images * Update repo name and regions * Update account number * Update framework name and check for None distribution * Add unit tests for smp v2 uri * Check enabled * Remove logging * Add cuda version in uri * Update cu121 * Update syntax * Fix black check * Fix black --------- Co-authored-by: huilgolr <yoda@ip-10-0-12-252.us-west-2.compute.internal> * Fix: Updated js mb compression logic - ModelBuilder (aws#4294) Co-authored-by: EC2 Default User <ec2-user@ip-172-16-54-104.us-west-2.compute.internal> * documentation: SMP v2 doc updates (aws#1423) (aws#4336) * doc update for estimator distribution art * add note to the SMP doc and minor fixes * remove subnodes * rm all v1 content as documenting everything in aws docs * fix build errors * fix white spaces * rm smdistributed from TF estimator distribution * rm white spaces * add notes to TF estimator distribution * fix links * incorporate feedback * update example values * fix version numbers in the notes Co-authored-by: Miyoung <cmiyoung@amazon.com> * prepare release v2.201.0 * update development version to v2.201.1.dev0 * Fix: Add additional model builder telemetry (aws#4334) * move telemetry code to public * add additional test --------- Co-authored-by: EC2 Default User <ec2-user@ip-172-16-54-104.us-west-2.compute.internal> * feature: support remote debug for sagemaker training job (aws#4315) * feature: support remote debug for sagemaker training job * change: Replace update_remote_config with 2 helper methods for enable and disable respectively * change: add new argument enable_remote_debug to skip set of test_jumpstart_estimator_kwargs_match_parent_class * chore: add jumpstart support for remote debug --------- Co-authored-by: Xinyu Xie <xixinyu@amazon.com> Co-authored-by: Evan Kravitz <evakravi@amazon.com> * Update tblib constraint (aws#4317) * Fix: Fix job_objective type (aws#4303) * change: update image_uri_configs 12-21-2023 08:32:41 PST * prepare release v2.202.0 * update development version to v2.202.1.dev0 * Using logging instead of prints (aws#4133) * documentation: update issue template. (aws#4337) * change: update model path in local mode (aws#4296) * Update model path in local mode * Add test * change: update image_uri_configs 12-22-2023 06:17:35 PST * prepare release v2.202.1 * update development version to v2.202.2.dev0 * change: create role if needed in `get_execution_role` (aws#4323) * Create role if needed in get_execution_role * Add tests * Change: More pythonic tags (aws#4327) * Change: More pythonic tags * Fix broken tags * More tags formatting and add a test * Fix tests * Raise Exception for debug (aws#4344) Co-authored-by: Ruilian Gao <ruiliann@amazon.com> * Change: Allow extra_args to be passed to uploader (aws#4338) * Change: Allow extra_args to be passed to uploader * Fix tests * Black * Fix test * Change: Drop py2 tag from the wheel as we don't support Python 2 (aws#4343) * Disable failed test in IR (aws#4345) * Disable failed test in IR * Fix format --------- Co-authored-by: Ruilian Gao <ruiliann@amazon.com> * change: update image_uri_configs 12-25-2023 06:17:33 PST * feat: Supporting tbac in load_run (aws#4039) * feature: support local mode in SageMaker Studio (aws#1300) (aws#4347) * feature: support local mode in SageMaker Studio * chore: fix typo * chore: fix formatting * chore: revert changes for docker compose logs * chore: black-format * change: Use predtermined dns-allow-listed-hostname for Studio Local Support * add support for CodeEditor and JupyterLabs --------- Co-authored-by: Erick Benitez-Ramos <141277478+benieric@users.noreply.github.com> Co-authored-by: Mufaddal Rohawala <mufi@amazon.com> * prepare release v2.203.0 * update development version to v2.203.1.dev0 * change: update image_uri_configs 12-29-2023 06:17:34 PST * query hf api for model md (aws#4346) Co-authored-by: EC2 Default User <ec2-user@ip-172-16-54-104.us-west-2.compute.internal> * fix: skip failing integs (aws#4348) Co-authored-by: Mufaddal Rohawala <mufi@amazon.com> * change: TGI 1.3.3 (aws#4335) * prepare release v2.203.1 * update development version to v2.203.2.dev0 * feat: parallelize notebook search utils, add new operators (aws#4342) * feat: parallelize notebook search utils * chore: raise exception in notebook utils if thread has error * chore: improve variable name * fix: not passing region to get jumpstart bucket * chore: add sagemaker session to notebook utils * chore: address PR comments * feat: add support for includes, begins with, ends with * fix: pylint * feat: private util for model eula key * fix: unit tests, use verify_model_region_and_return_specs in notebook utils * Revert "feat: private util for model eula key" This reverts commit e2daefc. * chore: add search keywords to header * fix: change ConditionNot incorrect property Expression to Condition (aws#4351) * fix: Huggingface glue failing tests (aws#4367) * fix: Huggingface glue failing tests * fix: Sphinx doc build failure * fix: Huggingface glue failing tests * fix: failing sphinx tests * fix: failing sphinx tests * fix: failing black check * fix: sphinx doc errors * fix: sphinx doc errors * sphinx * black-format * sphinx * sphinx * sphinx --------- Co-authored-by: Mufaddal Rohawala <mufi@amazon.com> Co-authored-by: Erick Benitez-Ramos <benieric@amazon.com> * fix: Add PyTorch 2.1.0 SM Training DLC to UNSUPPORTED_DLC_IMAGE_FOR_SM_PARALLELISM list (aws#4356) * add 2.1 unsupported smddp * formatting * feat: Support custom repack model settings (aws#4328) * change: update sphinx version (aws#4377) * change: update sphinx version * Update sphinx * change: Updates for DJL 0.26.0 release (aws#4366) * change: TGI NeuronX (aws#4375) * TGI NeuronX * Update * Update * fix: add warning message for job-prefixed pipeline steps when no job name is provided (aws#4371) Co-authored-by: svia3 <svia@amazon.com> * change: JumpStart - TLV region launch (aws#4379) * feat: add throughput management support for feature group (aws#4359) * feat: add throughput management support for feature group * documentation: add doc for feature group throughput config --------- Co-authored-by: Nilesh PS <psnilesh@amazon.com> * change: Enable galactus integ tests (aws#4376) * feat: Enable galactus integ tests * fix flake8 * fix doc8 * trying to see if it works with slow tests * small fixes in import error * fix missing import * try to remove some dependencies from requirement to see if pr test can be fixed * fix flake8 * Enable more tests * Add rerun annotation and further remove dependencies * comment out 2 integ tests * Remove local mode test for now * fix flake8 * prepare release v2.204.0 * update development version to v2.204.1.dev0 * fix: Add validation for empty ParameterString value in start local pipeline (aws#4354) * feat: Support selective pipeline execution for function step (aws#4372) * change: update image_uri_configs 01-24-2024 06:17:33 PST * fix: update get_execution_role_arn from metadata file if present (aws#4388) * fix: Support using PipelineDefinitionConfig in local mode (aws#4352) * fix: remove fastapi and uvicorn dependencies (aws#4365) They are not used in the codebase. Closes aws#4361 aws#4295 * prepare release v2.205.0 * update development version to v2.205.1.dev0 * change: TGI NeuronX 0.0.17 (aws#4390) * fix: Support PipelineVariable for ModelQualityCheckConfig attributes (aws#4353) * feat: Logic to detect hardware GPU count and aggregate GPU memory size in MiB (aws#4389) * Add logic to detect hardware GPU count and aggregate GPU memory size in MiB * Fix all formatting * Addressed PR review comments * Addressed PR Review messages * Addressed PR Review Messages * Addressed PR Review comments * Addressed PR Review Comments * Add integration tests * Add config * Fix integration tests * Include Instance Types GPU infor Config files * Addressed PR review comments * Fix unit tests * Fix unit test: 'Mock' object is not subscriptable --------- Co-authored-by: Jonathan Makunga <makung@amazon.com> * fix: fixed create monitoring schedule failing after validation error (aws#4385) Co-authored-by: Keshav Chandak <chakesh@amazon.com> * Add collection type support for Feaure Group Ingestion. Add TargetStores support for PutRecord and Ingestion. * Remove merge conflicts. * Update the feature definition type * Black formatting * Fix Flake8 formatting * Fix Pylint * Fix Formatting. --------- Co-authored-by: sagemaker-bot <sagemaker-bot@amazon.com> Co-authored-by: ci <ci> Co-authored-by: qidewenwhen <32910701+qidewenwhen@users.noreply.github.com> Co-authored-by: Keshav Chandak <keshav.chandak1995@gmail.com> Co-authored-by: Keshav Chandak <chakesh@amazon.com> Co-authored-by: stacicho <stacicho@amazon.com> Co-authored-by: Teng-xu <67929972+Teng-xu@users.noreply.github.com> Co-authored-by: huilgolr <yoda@ip-10-0-12-252.us-west-2.compute.internal> Co-authored-by: Gary Wang <38331932+gwang111@users.noreply.github.com> Co-authored-by: EC2 Default User <ec2-user@ip-172-16-54-104.us-west-2.compute.internal> Co-authored-by: akrishna1995 <38850354+akrishna1995@users.noreply.github.com> Co-authored-by: Miyoung <cmiyoung@amazon.com> Co-authored-by: Xinyu Xie <xiexinyucrab@126.com> Co-authored-by: Xinyu Xie <xixinyu@amazon.com> Co-authored-by: Evan Kravitz <evakravi@amazon.com> Co-authored-by: martinRenou <martin.renou@gmail.com> Co-authored-by: Duc Trung Le <leductrungxf@gmail.com> Co-authored-by: ruiliann666 <141953824+ruiliann666@users.noreply.github.com> Co-authored-by: Ruilian Gao <ruiliann@amazon.com> Co-authored-by: ananth102 <ananthbashyam1@gmail.com> Co-authored-by: Mufaddal Rohawala <89424143+mufaddal-rohawala@users.noreply.github.com> Co-authored-by: Erick Benitez-Ramos <141277478+benieric@users.noreply.github.com> Co-authored-by: Mufaddal Rohawala <mufi@amazon.com> Co-authored-by: amzn-choeric <105388439+amzn-choeric@users.noreply.github.com> Co-authored-by: evakravi <69981223+evakravi@users.noreply.github.com> Co-authored-by: Erick Benitez-Ramos <benieric@amazon.com> Co-authored-by: Sirut Buasai <73297481+sirutBuasai@users.noreply.github.com> Co-authored-by: Sindhu Somasundaram <56774226+sindhuvahinis@users.noreply.github.com> Co-authored-by: Stephen Via <51342648+svia3@users.noreply.github.com> Co-authored-by: svia3 <svia@amazon.com> Co-authored-by: Haixin Wang <98612668+haixiw@users.noreply.github.com> Co-authored-by: Nilesh PS <nps17thatsme@gmail.com> Co-authored-by: Nilesh PS <psnilesh@amazon.com> Co-authored-by: jiapinw <95885824+jiapinw@users.noreply.github.com> Co-authored-by: Jay Goyani <135654128+jgoyani1@users.noreply.github.com> Co-authored-by: Justin <justinm088@hotmail.com> Co-authored-by: Jonathan Makunga <54963715+makungaj1@users.noreply.github.com> Co-authored-by: Jonathan Makunga <makung@amazon.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
feature: support local mode in SageMaker Studio
chore: fix typo
chore: fix formatting
chore: revert changes for docker compose logs
chore: black-format
change: Use predtermined dns-allow-listed-hostname for Studio Local Support
add support for CodeEditor and JupyterLabs
Issue #, if available:
Description of changes:
Testing done:
Merge Checklist
Put an
xin the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.General
Tests
unique_name_from_baseto create resource names in integ tests (if appropriate)By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.