mpfshell

2016-06-21, sw@kaltpost.de

A simple shell based file explorer for ESP8266 and WiPy Micropython based devices.

The shell is a helper for up/downloading files to the ESP8266 (over serial line and Websockets) and WiPy (serial line and telnet). It basically offers commands to list and upload/download files on the flash FS of the device.

Main features:

• Support for serial connections (ESP8266 and WiPi)
• Support for websockets (via WebREPL) connections (ESP8266 only)
• Support for telnet connections (WiPy only)
• Full directory handling (enter, create, remove)
• Transfer (upload/download) of multiple files matching a reg.-exp.
• All files are transferred in binary mode, and thus it should be possible to also upload pre-compiled code (.mpy) too.
• Integrated REPL (supporting a workflow like: upload changed files, enter REPL, test, exit REPL, upload ...)
• Fully scriptable
• Tab-completion
• Command history
• Best of all: it comes with color

Note: The software is mainly tested on Ubunto 16.04 LTS. However, there is basic Windows support (tested with Python 3.5 and PySerial 3.1), but some of the keys (e.g. Tab are note working as expected yet).

Requirements

General:

• ESP8266 or WiPy board running latest Micropython
• For the ESP8266 firware build from the repository, please not, that WebREPL is not started by default. For more information see the quickstart.
• For the WiPy, please note, that you need to enable REPL on UART if you intend to connect via serial line to the WiPy (see here)

For the shell:

• Python >= 2.7 or Python >= 3.4
• The PySerial library >= 2.7 (sudo pip install pyserial)
• The colorama library >= 0.3.6 (sudo pip install colorama)
• The websocket-client library >= 0.35.0 (sudo pip install websocket-client)

IMPORTANT: PySerial versions before 2.7 really don't work!!! It is highly recommended to use PySerial version 3.x on Python2 and Python3.

Note: The tools only works if the REPL is accessible on the device!

Installing

To install this tool for Python 2, execute the following:

sudo pip install pyserial
sudo pip install colorama
sudo pip install websocket_client
sudo python setup.py install

To install this tool for Python 3, execute the following:

sudo pip3 install pyserial
sudo pip3 install colorama
sudo pip3 install websocket_client
sudo python3 setup.py install

Known Issues

• For PySerial 2.6 the REPL is deactivated since Miniterm which comes with 2.6 seams broken.

General

TAB Completion

The shell supports TAB completion for commands and file names. So it totally is worth it pressing TAB-TAB every now and then :-)

File/Directory Names

File-names including whitespaces are supported, but such names need to be enclosed in quotes. E.g. accessing a file named "with white space.txt" needs to quoted:

get "with white space.txt"
put "with white space.txt" without-white-space.txt
put without-white-space.txt "with white space.txt"

The following characters are accepted for file and directory names:

A-Za-z0-9 _%#~@/\$!\*\.\+\-

Shell Usage

Note: Since version 0.7.1, the shell offers caching for file and directory names. It is now enabled by default. To disable caching, add the --nocache flag on the command line.

Start the shell with:

mpfshell

At the shell prompt, first connect to the device. E.g. to connect via serail line:

mpfs> open ttyUSB0

Or connect via websocket (ESP8266 only):

mpfs> open ws:192.168.1.1,python

Or connect vial telnet (WiPy only):

mpfs> open tn:192.168.1.1,micro,python

Note: Login and password are optional. If left out, they will be asked for.

Now you can list the files on the device with:

mpfs> ls

To upload e.g. the local file "boot.py" to the device use:

mpfs> put boot.py

If you like to use a different filename on the device, you could use this:

mpfs> put boot.py main.py

Or to upload all files that match a regular expression from the current local directory to the current remote directory:

mpfs> mput .*\.py

And to download e.g. the file "boot.py" from the device use:

mpfs> get boot.py

Using a different local file name:

mpfs> get boot.py my_boot.py

Or to download all files that match a regular expression from the current remote directory to the current local directory:

mpfs> mget .*\.py

To remove a file (or directory) on the device use:

mpfs> rm boot.py

Or remove all remote files that match a regular expression:

mpfs> mrm test.*\.py

To create a new remote directory:

mpfs> md test

To navigate remote directories:

mpfs> cd test
mpfs> cd ..
mpfs> cd /some/full/path

See which is the current remote directory:

mpfs> pwd

Remove a remote directory:

mpfs> rm test

Note: The directory to delete needs to be empty!

To navigate on the local filesystem, use:

lls, lcd lpwd

Enter REPL:

repl

To exit REPL and return to the file shell use Ctrl+Alt+]

For a full list of commands use:

mpfs> help
mpfs> help <command>

The shell is also scriptable.

E.g. to execute a command, and then enter the shell:

mpfshell -c "open ttyUSB0"

Or to copy the file "boot.py" to the device, and don't enter the shell at all:

mpfshell -n -c "open ttyUSB0; put boot.py"

It is also possible to put a bunch of shell commands in a file, and then execute them from that file.

E.g. creating a file called "myscript.mpf":

open ttyUSB0 
put boot.py
put main.py
ls

And execute it with:

mpfshell -s myscript.mpf    

Running the Shell in a Virtual Environment

Somtimes it is the easiest way to setup a virtual environment to satisfy the requirements. E.g. on Debian Jessi (which still has PySerial 2.6), this could be done like so (assuming you are within the mpfshell base directory:

Install support for virtual environments:

sudo apt-get install python3-venv

Create a new virtual environment:

pyvenv venv

Activate it (so ervery following pip3 install goes to the new virtuel environment):

source venv/bin/activate

Now install the dependencies to the virtual environment:

pip3 install pyserial
pip3 install colorama
pip3 install websocket_client

Now run the shell with the following command:

python3 -m mp.mpfshell

Note: The environment has always be activated with the above command.