Skip to content
This repository has been archived by the owner on Jun 6, 2024. It is now read-only.

Latest commit

 

History

History
104 lines (78 loc) · 4.87 KB

README.md

File metadata and controls

104 lines (78 loc) · 4.87 KB
Raw

OpenPAI VS Code Client

OpenPAI Client is an extension to connect PAI clusters, submit AI jobs, and manage files on HDFS.

Get Started

  1. Run command 'PAI: Add PAI Cluster' from command palette (Ctrl+Shift+P)
  2. Fill in username, password and other fields and press "finish" button.

  1. Your cluster will be shown in PAI CLUSTER EXPLORER (at the bottom of VS Code's file explorer by default)
  2. Expand the tree node and double click the command you like.

Submit Job

The extension uses a job config file (in json format) to describe a traing job. You can create multiple job config files for your code so that you can choose proper settings upon job submission.

  1. Generate a PAI job config file by:

    • Double click "Create Job Config..."

    or

    • Right click a source code file and click "Create PAI Job Config JSON"

  2. Fill the created PAI job config JSON's command and job name fields.

  3. Right click the created JSON file and select "Submit Job to PAI Cluster"

  4. The job will be submitted to PAI cluster

Simulate Job Running

The extension has a useful feature called "Simulate Job Running". It enables the users to run and debug their AI jobs in local environment without really submitting the job to PAI cluster. This is helpful when the users want to verify the docker image, start up command, and the code quickly.

  1. Right click a job config file and click "Simulate PAI Job Running" The extension will prepare necessary files and create a docker file in local folder for later use.

  2. A message box pops up, users have two options:

    • Simulate first task in VS Code terminal

    This will execute the simulation command within VS Code terminal window.

    • Reveal in Explorer

    This will pops up the OS's file explorer and navigates to the simulation folder. Users can manually execute the simulation command. (run-docker.cmd on Windows, or run-docker.sh on Linux).

Sidebar

HDFS Explorer and Job List view will be shown in the extension's view container (sidebar).

HDFS Explorer

  1. You are able to connect to an OpenPAI cluster's HDFS by double click its corresponding tree node.
  2. The HDFS directory structure will be expended in the tree view.
  3. Right click the folder or file node to perform file system operations.

Job List

  1. Browse latest jobs in the OpenPAI cluster in job list tree view.
  2. Browse recent submitted jobs by local (extension) in job list tree view.
  3. Job list will auto refresh every 10 seconds.
  4. Double click job node to browse Job details in external browser.

Commands

Command Pallete

Name Description
PAI: Add PAI Cluster Add a new cluster to PAI Cluster Explorer
PAI: Open Dashboard Open dashboard pages in VS Code
PAI: Submit Job to PAI Cluster Select a PAI job config file and submit it to PAI cluster
PAI: Create PAI Job Config File Create an empty PAI job config file
PAI: Simulate PAI Job Running Generate Dockerfile to simulate PAI job running

PAI Cluster Explorer

Name Description
Open Web Portal... Open PAI's web portal page in VS Code (right click to open in browser)
List Jobs... Open PAI's job list page in VS Code (right click to open in browser)
Create Job Config... Create an empty PAI job config file
Submit Job... Submit job to selected cluster
Simulate Job Running... Generate Dockerfile to simulate PAI job running
Edit Configuration... Edit cluster configuration
Open HDFS... Open HDFS explorer of selected cluster

Settings

ID Description
pai.job.upload.enabled Controls whether the extension will upload your project files to PAI job config's code dir automatically
pai.job.upload.exclude Glob pattern for excluding files and folders
pai.job.upload.include Glob pattern for including files and folders
pai.job.generateJobName.enabled Controls whether the extension will add a random suffix to your job name when submitting job
pai.job.jobList.recentJobsLength Controls the number of recently submitted jobs to keep in history for each PAI cluster
pai.job.jobList.allJobsPageSize Controls the page size of list when listing jobs for each PAI cluster
pai.job.jobList.refreshInterval Controls the refresh interval of job list (in seconds)
pai.hdfs.location Location where hdfs explorer will be shown

Requirements

PAI Cluster Version >= 0.8.0

Contributing

https://github.com/Microsoft/pai#how-to-contribute

Report an issue:

If you have issue/ bug/ new feature request, please submit it at GitHub

License

MIT