Skip to content
This repository has been archived by the owner on Jun 6, 2024. It is now read-only.

[BUG] Fix hadoop ai build path #2262

Merged
merged 3 commits into from
Mar 5, 2019
Merged

[BUG] Fix hadoop ai build path #2262

merged 3 commits into from
Mar 5, 2019

Conversation

mzmssg
Copy link
Member

@mzmssg mzmssg commented Mar 4, 2019

Issue link: #2021

@mzmssg mzmssg requested review from hao1939 and ydye March 4, 2019 08:25
@coveralls
Copy link

Coverage Status

Coverage decreased (-0.3%) to 52.374% when pulling 41acd13 on zimiao/fix_hadoop_build_path into 659e155 on master.

@coveralls
Copy link

coveralls commented Mar 4, 2019 •
edited
Loading

Coverage Status

Coverage decreased (-7.9%) to 52.627% when pulling 00ad83e on zimiao/fix_hadoop_build_path into d150ced on master.

ydye
ydye reviewed Mar 4, 2019
View reviewed changes
src/hadoop-ai/build/build-pre.sh Outdated
@@ -42,7 +42,7 @@ echo "hadoopbinarypath:${hadoopBinaryDir}"
}

# When Changing the patch id, please update the filename here.
rm ${hadoopBinaryDir}/*-done
rm ${hadoopBinaryDir}*-done
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For which case, the removed line won't work?

Copy link
Member Author

@mzmssg mzmssg Mar 4, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems misleaded by the error message:

hadoopbinarypath:/hadoop-binary/
rm: cannot remove '/hadoop-binary//*-done': No such file or directory
Sending build context to Docker daemon 16.9kB

Confirmed that double slashes should work. Need to investigate more about origin issue.

Whatever, removing redundent slash isn't bad.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this change handle this path in shell script?

/hadoop-binary/*-done

@mzmssg mzmssg merged commit 2429b23 into master Mar 5, 2019
sunqinzheng added a commit that referenced this pull request Mar 20, 2019
@sunqinzheng
[VS Code] view container (#2301)
021a645 
* add a dashboard in grafana to list all tasks in node (#2197)

* Fix format in issue templates (#2233)

Fix format in issue templates:
- remove trailing spaces
- change chinese colon into english

* Fix auto retries when out of memory. (#1108)

* Distinguish cgroup OOM from dmesg.

* Remove cgroup OOM detection

Make all OOM cause exiting by 5

* Exit 55 when OOM

* Refine homepage for new users (#2155)

Updated first level bullets, to add more content for administrators and users, who is first time touch OpenPAI, or computing platform.

* Fix yarn container failed when docker container exited quickly. (#2256)

* REST server: remove expires in JWT payload of unit test (#2263)

* Deploy: add explicit config field in webportal  plugin (#2251)

* Deploy: add explicit config field in webportal  plugin

* Fix json.dumps

* t

* fix

* Update PLUGINS.md

* Update webportal.md

* alert on unhealthy gpu (#2209)

* Pylon: fix double start query in yarn redirect (#2258)

* Pylon: fix double start query in yarn redirect

* Hide debug info in docker-compose.yaml

* adapt user transfer script to new config (#2266)

* Webportal: add pai-version attribute to <pai-plugin> (#2245)

* Webportal: add pai-version attribute to <pai-plugin>

* Use preprocess to apply window.PAI_VERSION

* set version in layout.html

* Fix ib drivers bug (#2269)

* FIx ib installation script bug (#2271)

* [BUG] Fix hadoop ai build path (#2262)

* fix hadoop ai build bugs

* refine

* Web portal submit job: support init json from sessionStorage. (#2253)

* YARN and HDFS log persistence  (#2244)

* rm log persist

* change log dir to host

* persist nm log to host

* resolve conflict

* persist namenode log

* persist data node log

* add comments

* move log path to common pai storage

* use twisted in yarn-exporter (#2273)

* [Job Debugging] Basic Implement Of Job Debugging. (#2272)

* Refine document for new user to submit job (#2278)

1. add new guidance to submit job for beginners.
2. refine homepage to connect with new guidance.
3. reorganize content of troubleshooting for next refactoring.
4. fix links in md files.

* [Drivers] Fix the issue when installing IB drivers.  (#2275)

* fix can not report zombie process using gpu error (#2279)

* fix external process error

* add debug log

* fix short ID and long ID do not match

* use time based atomic ref to exchange info between threads

* add test case for AtomicRef

* fix bug in file remove (#2288)

* fix hadoop build error (#2296)

* export vc/node related metrics from yarn (#2289)

* 720

* open hdfs explorer in view container
enable tslint rule "ordered-imports"

* add tslint rule for indent

* add home button to hdfs explorer's navigation;
adjust octicon's color

* fix lint error

* [VS Code] Add job list (#2160)

* add job list view to pai extension

* [VS Code] joblist fix (#2185)

* eager load recent jobs when job submitted

* avoid eager getChildren, and let vscode treeview.reveal do it implicitly

* fix lint error
@mzmssg mzmssg deleted the zimiao/fix_hadoop_build_path branch March 28, 2019 08:12
sunqinzheng added a commit that referenced this pull request Mar 28, 2019
@sunqinzheng
[VS Code] view container (#2301)
f1b1535 
* add a dashboard in grafana to list all tasks in node (#2197)

* Fix format in issue templates (#2233)

Fix format in issue templates:
- remove trailing spaces
- change chinese colon into english

* Fix auto retries when out of memory. (#1108)

* Distinguish cgroup OOM from dmesg.

* Remove cgroup OOM detection

Make all OOM cause exiting by 5

* Exit 55 when OOM

* Refine homepage for new users (#2155)

Updated first level bullets, to add more content for administrators and users, who is first time touch OpenPAI, or computing platform.

* Fix yarn container failed when docker container exited quickly. (#2256)

* REST server: remove expires in JWT payload of unit test (#2263)

* Deploy: add explicit config field in webportal  plugin (#2251)

* Deploy: add explicit config field in webportal  plugin

* Fix json.dumps

* t

* fix

* Update PLUGINS.md

* Update webportal.md

* alert on unhealthy gpu (#2209)

* Pylon: fix double start query in yarn redirect (#2258)

* Pylon: fix double start query in yarn redirect

* Hide debug info in docker-compose.yaml

* adapt user transfer script to new config (#2266)

* Webportal: add pai-version attribute to <pai-plugin> (#2245)

* Webportal: add pai-version attribute to <pai-plugin>

* Use preprocess to apply window.PAI_VERSION

* set version in layout.html

* Fix ib drivers bug (#2269)

* FIx ib installation script bug (#2271)

* [BUG] Fix hadoop ai build path (#2262)

* fix hadoop ai build bugs

* refine

* Web portal submit job: support init json from sessionStorage. (#2253)

* YARN and HDFS log persistence  (#2244)

* rm log persist

* change log dir to host

* persist nm log to host

* resolve conflict

* persist namenode log

* persist data node log

* add comments

* move log path to common pai storage

* use twisted in yarn-exporter (#2273)

* [Job Debugging] Basic Implement Of Job Debugging. (#2272)

* Refine document for new user to submit job (#2278)

1. add new guidance to submit job for beginners.
2. refine homepage to connect with new guidance.
3. reorganize content of troubleshooting for next refactoring.
4. fix links in md files.

* [Drivers] Fix the issue when installing IB drivers.  (#2275)

* fix can not report zombie process using gpu error (#2279)

* fix external process error

* add debug log

* fix short ID and long ID do not match

* use time based atomic ref to exchange info between threads

* add test case for AtomicRef

* fix bug in file remove (#2288)

* fix hadoop build error (#2296)

* export vc/node related metrics from yarn (#2289)

* 720

* open hdfs explorer in view container
enable tslint rule "ordered-imports"

* add tslint rule for indent

* add home button to hdfs explorer's navigation;
adjust octicon's color

* fix lint error

* [VS Code] Add job list (#2160)

* add job list view to pai extension

* [VS Code] joblist fix (#2185)

* eager load recent jobs when job submitted

* avoid eager getChildren, and let vscode treeview.reveal do it implicitly

* fix lint error
qfyin pushed a commit that referenced this pull request Apr 1, 2019
@sunqinzheng @qfyin
[VS Code] version 0.2.0 (#2442)
efdca68 
* add installation guide for VS code extension (#2223)

* add installation guide for VS code extension

* [VS Code] view container (#2301)

* add a dashboard in grafana to list all tasks in node (#2197)

* Fix format in issue templates (#2233)

Fix format in issue templates:
- remove trailing spaces
- change chinese colon into english

* Fix auto retries when out of memory. (#1108)

* Distinguish cgroup OOM from dmesg.

* Remove cgroup OOM detection

Make all OOM cause exiting by 5

* Exit 55 when OOM

* Refine homepage for new users (#2155)

Updated first level bullets, to add more content for administrators and users, who is first time touch OpenPAI, or computing platform.

* Fix yarn container failed when docker container exited quickly. (#2256)

* REST server: remove expires in JWT payload of unit test (#2263)

* Deploy: add explicit config field in webportal  plugin (#2251)

* Deploy: add explicit config field in webportal  plugin

* Fix json.dumps

* t

* fix

* Update PLUGINS.md

* Update webportal.md

* alert on unhealthy gpu (#2209)

* Pylon: fix double start query in yarn redirect (#2258)

* Pylon: fix double start query in yarn redirect

* Hide debug info in docker-compose.yaml

* adapt user transfer script to new config (#2266)

* Webportal: add pai-version attribute to <pai-plugin> (#2245)

* Webportal: add pai-version attribute to <pai-plugin>

* Use preprocess to apply window.PAI_VERSION

* set version in layout.html

* Fix ib drivers bug (#2269)

* FIx ib installation script bug (#2271)

* [BUG] Fix hadoop ai build path (#2262)

* fix hadoop ai build bugs

* refine

* Web portal submit job: support init json from sessionStorage. (#2253)

* YARN and HDFS log persistence  (#2244)

* rm log persist

* change log dir to host

* persist nm log to host

* resolve conflict

* persist namenode log

* persist data node log

* add comments

* move log path to common pai storage

* use twisted in yarn-exporter (#2273)

* [Job Debugging] Basic Implement Of Job Debugging. (#2272)

* Refine document for new user to submit job (#2278)

1. add new guidance to submit job for beginners.
2. refine homepage to connect with new guidance.
3. reorganize content of troubleshooting for next refactoring.
4. fix links in md files.

* [Drivers] Fix the issue when installing IB drivers.  (#2275)

* fix can not report zombie process using gpu error (#2279)

* fix external process error

* add debug log

* fix short ID and long ID do not match

* use time based atomic ref to exchange info between threads

* add test case for AtomicRef

* fix bug in file remove (#2288)

* fix hadoop build error (#2296)

* export vc/node related metrics from yarn (#2289)

* 720

* open hdfs explorer in view container
enable tslint rule "ordered-imports"

* add tslint rule for indent

* add home button to hdfs explorer's navigation;
adjust octicon's color

* fix lint error

* [VS Code] Add job list (#2160)

* add job list view to pai extension

* [VS Code] joblist fix (#2185)

* eager load recent jobs when job submitted

* avoid eager getChildren, and let vscode treeview.reveal do it implicitly

* fix lint error

* [VS Code] default to generate jsonc job config file  (#2368)

* 720

* open hdfs explorer in view container
enable tslint rule "ordered-imports"

* add tslint rule for indent

* add home button to hdfs explorer's navigation;
adjust octicon's color

* fix lint error

* [VS Code] Add job list (#2160)

* add job list view to pai extension

* [VS Code] joblist fix (#2185)

* eager load recent jobs when job submitted

* avoid eager getChildren, and let vscode treeview.reveal do it implicitly

* default to generate jsonc job config file

* [VS Code] Refine error messages; Fix Cluster Explorer's bug

* [VS Code] changelog and readme (#2429)

* 539

* 712

* 452

* [VS Code] v0.11 compatible issue (#2457)

* 536

* 600

* [VS Code] fix cluster explorer's right-click menu (#2463)
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@ydye ydye ydye approved these changes

@fanyangCS fanyangCS fanyangCS approved these changes

@hao1939 hao1939 Awaiting requested review from hao1939

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@mzmssg @coveralls @fanyangCS @ydye