Run an experiment at local met problem！

[wenku1992abc]

According to GetStarted.md ,as follws,

Install NNI through source code

git clone -b v0.1 https://github.com/Microsoft/nni.git
cd nni
chmod +x install.sh
source install.sh

I had sucessfully installed NNI ,But when I run an experiment at local ,do such commad: nnictl create --config ~/nni/examples/trials/mnist-annotation/config.yml ,It shows an error,as follws,

Info: Checking experiment...
Info: Starting restful server...
Info: Checking restful server...
Error: Restful server start failed!

Could you help me?

[QuanluZhang]

1. Please make sure that port 51188 is not occupied by other processes.
2. Then please run 'nnictl log stderr' to see the error message.
3. You can also use 'ps aux | grep node' to check is there any running rest server process.

[wenku1992abc]

1.Check port 51188 is not occupied:

2. Run 'nnictl log stderr' ,It shows,as follws:
   (node:18566) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
   /home/user/.local/bin/nnimanager: line 3: 18566 Terminated /home/user/.local/nni/node/bin/node main.js $@
   (node:31077) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
   /home/user/.local/bin/nnimanager: line 3: 31077 Terminated /home/user/.local/nni/node/bin/node main.js $@
   (node:40883) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
   /home/user/.local/bin/nnimanager: line 3: 40883 Terminated /home/user/.local/nni/node/bin/node main.js $@
   (node:870) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
   /home/user/.local/bin/nnimanager: line 3: 870 Terminated /home/user/.local/nni/node/bin/node main.js $@
   (node:31529) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
   /home/user/.local/bin/nnimanager: line 3: 31529 Terminated /home/user/.local/nni/node/bin/node main.js $@

3.Run 'ps aux | grep node',It shows,as follws:
user 35104 0.0 0.0 14536 1020 pts/26 S+ 09:52 0:00 grep --color=auto node

[QuanluZhang]

Those lines are only warning (we will fix this warning later).
Could you run:
nnictl stop
then, nnictl create --config ~/nni/examples/trials/mnist-annotation/config.yml.

If there is still error, please post the content of the following three files:
~/nni/nnictl/log/stderr
~/nni/nnictl/log/stdout
~/nni/experiments//log/nnimanager.log
Thanks.

[wenku1992abc]

@QuanluZhang Hi,QuanluZhang.Thanks your reply

• Run 'nnictl stop',it shows,as follows:
  Info: Stoping experiment...
  Info: Experiment is not running...

Then run 'nnictl create --config ~/nni/examples/trials/mnist-annotation/config.yml.',it shows same error as before

• Post the content of the following three files

1.The content of ~/nni/nnictl/log/stderr ,as follows:
(node:18566) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
/home/user/.local/bin/nnimanager: line 3: 18566 Terminated /home/user/.local/nni/node/bin/node main.js $@
(node:31077) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
/home/user/.local/bin/nnimanager: line 3: 31077 Terminated /home/user/.local/nni/node/bin/node main.js $@
(node:40883) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
/home/user/.local/bin/nnimanager: line 3: 40883 Terminated /home/user/.local/nni/node/bin/node main.js $@
(node:870) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
/home/user/.local/bin/nnimanager: line 3: 870 Terminated /home/user/.local/nni/node/bin/node main.js $@
(node:31529) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
/home/user/.local/bin/nnimanager: line 3: 31529 Terminated /home/user/.local/nni/node/bin/node main.js $@
(node:3118) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
/home/user/.local/bin/nnimanager: line 3: 3118 Terminated /home/user/.local/nni/node/bin/node main.js $@
(node:26800) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
/home/user/.local/bin/nnimanager: line 3: 26800 Terminated /home/user/.local/nni/node/bin/node main.js $@
(node:12632) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
/home/user/.local/bin/nnimanager: line 3: 12632 Terminated /home/user/.local/nni/node/bin/node main.js $@

2.The content of ~/nni/nnictl/log/stdout is empty!

3.The content of ~/nni/experiments//log/nnimanager.log ,as follows:
[2018-09-17T03:33:23.587Z] INFO Rest server listening on: http://0.0.0.0:51188

[yds05]

Hi, @wenku1992abc, thanks for reporting the issue and providing the logs.

Could you please help confirm if the log you pasted from your ~/nni/experiments//log/nnimanager.log is a complete log?

A normal log contents in ~/nni/experiments//log/nnimanager.log is something like:
[2018-09-17T03:36:53.950Z] INFO Rest server listening on: http://0.0.0.0:51188
[2018-09-17T03:36:56.773Z] INFO GET: /check-status: body:
{}
[2018-09-17T03:37:03.802Z] INFO GET: /trial-jobs: body:
{}
....................................................
....................................................

if the log from ~/nni/experiments//log/nnimanager.log is a complete one, could you also check if port 51188 is opened on your machine's firewall?

If still have issue, feel free to let us know. Thanks.

[wenku1992abc]

@yds05 My pleasure!
1.Yes, I checked it again! I am sure log from ~/nni/experiments//log/nnimanager.log is a complete one!

2.According to command 'rpm -qa|grep iptables' in linux, I checked if my machine installed firewall or not.The results is that it has no firewall

[wenku1992abc]

@QuanluZhang @yds05 May be my problem is the same as this issue,#85 (comment)
my python is also installed with anaconda

[yds05]

@wenku1992abc Yes, probably it's related to that issue caused by python installed via anaconda. We will fully test nni on anaconda python soon, and let you know if we have any conclusion/solution. Thanks!

[scarlett2018]

@wenku1992abc - had you verified whether the resolution works for you? the support had been released in Sept 28.

[scarlett2018]

I'm closing the item as the fix had been released, @wenku1992abc please feel free to reopen the issue if it still not work for you, thanks a lot.

[wenku1992abc]

@scarlett2018 Hi,I use new nni source code to install nnictl,after installed
when i use such command:
nnictl create --config ./examples/trials/mnist/config.yml --port 8085

the output is :

INFO: Starting restful server...
ERROR: Restful server start failed!
INFO: Stdout:

            Experiment start time 2018-11-30 15:23:19

---

INFO: Stderr:

            Experiment start time 2018-11-30 15:23:19

---

[wenku1992abc]

~/nni/experiments//log/nnimanager.log :

[2018-11-30T07:10:12.983Z] INFO Rest server listening on: http://0.0.0.0:8085
[2018-11-30T07:10:12.970Z] INFO RestServer start
[2018-11-30T07:10:12.978Z] INFO RestServer base port is 8085
[2018-11-30T07:10:27.643Z] INFO Experiment done, cleaning up...