Skip to content

PyMoIP Gateway

Jump to bottom
64rulez at gmail edited this page Sep 14, 2021 · 18 revisions

PyMoIP Gateway Wiki

0. Prerequisites and setup

python3 3.7 or more

pyserial module :

python3 -m pip install pyserial

websockets module :

python3 -m pip install

Recommended : Telnet (for display console)

sudo apt-get install telnet

Setup :

  • Create a 'BASEDIR' folder, such as '/home/pi/python/PyMoIP'

  • In 'BASEDIR' folder, create a 'Gateway' folder

  • Copy the following files to 'BASEDIR'/Gateway

    __main__.py GatewayServer.py Session.py biject.py

  • If necessary, edit the following files in 'BASEDIR'/Gateway

    GatewayConfiguration.json neverbanned.json

1. Usage

(from 'BASEDIR')

usage: python3 -m Gateway [-h] [--ws PORT] [--tcp PORT] [--display PORT] [-i IP] [-c URI] [-v URI] [-p PING] [-b PROTO] [-n SWITCH] [-g PATH] [-l PATH]

Launch the PyMoIP gateway.

Arguments:

  • -h, --help show this help message and exit
  • --ws PORT Specify a port for a WebSocket Server. (Default=9001)

Note : If no ws port is provided, no WS Server will be created.

  • --tcp PORT Specify a port for a TCP Server. (Default=9000)

Note : If no tcp port is provided, no TCP Server will be created.

  • --display PORT Specify a port for a TCP display Server. (Default=8999)
  • -i IP, --ip IP Specify an IP address to listen to. (Default=0.0.0.0 (all @IPs))
  • -c URI, --command URI Specify an URI for WS Gatewaylink to 'Télétel' command channel (default='ws://localhost:8764' for normal command channel)
  • -v URI, --videotext URI Specify an URI to WS default videotext server (Default=ws://localhost:8765)
  • -p PING, --interval PING Specify the ping interval to WS default videotext server (Default=None)
  • -b PROTO, --subprotocols PROTO Specify subprotocols for WS default videotext server (Default=[])
  • -n SWITCH, --nocommandchannel SWITCH Disable command channel
  • -g PATH, --configpath PATH Path to configuration file (with a tailling '/') (must be RW, defaults to './Gateway/')
  • -l PATH, --logpath PATH Path to log file (with a tailling '/') (must be RW, defaults to '/home/pi/PyMoIP_dir/')

It is recommended to launch PyMoIP programs [Gateway, Server] with the script 'startvdt.sh'.

It is recommended to end PyMoIP programs [Gateway, Server] with the script 'stopvdt.sh'.

It is recommended to restart PyMoIP programs [Gateway, Server] with the script 'restartvdt.sh'.

[To be complemented]

2. Known bugs

So far, not much as I'm trying to fix bugs as soon as they're discovered but many features are still "wet" and had only few functionnal testing.

Bug #1 (recurrent) : "Rogue port scanners"

I've observed time to time some random crashes or hang of either 'PyMoIP Gateway' or 'PyMoIP Server' caused by port scanners. Some safety measures have been implemented in order to prevent those as :

  • Trace any received byte per launch/session. This is optionnal as it needs some proper maintenance and voids any (possible ?) confidentiality, but it will help in further diagnostics.
  • Automatic @IP ban, when sessions are vibily not human (too short or typing too fast). There's a risk of false positive. To force the unban of an @IP, it's necessary to stop 'PyMoIP Gateway' and manually edit the relevant .JSON file. [currently, about 200 @IP are banned]. Automatic unban after a delay (by default, 100 days).
  • Invalid unicode received as payload of port scanners. Try to avoid raw printing of received data but there might be some places in the code [PyMoIP Gateway and/or Server] where those debug points remain. That's difficult to exclusively track this down. [While videotext is mostly 7 bits characters, PyMoIP Gateway is agnostic to parity and must support transparently 8 bits characters to allow passing photovideotext]

Bug #2 (known) : "Redirected Local/Line"

In a Teletel scenario, a user will type in a service short name and will be redirected to that videotext service. During this time, while the user session is redirecting/redirected to a 'remote' videotext service, that initial user session to the Teletel server is maintained in order to simulate the virtual billing. The gateway needs to monitor the [local/Line] function key in order to end the 'remote' videotext service session and revert the user to it's initial Teletel session. The gateway also needs to monitor the [local/Line] function key in order to end the 'Teletel' videotext service session while the user is not redirected.

Due to the asynchronous and reentrant nature of the code, if the user press [Local/Line] immediately after <service_code>[Send] (whatever the delay or the redirecting/redirected state), it'll be wrongly interpreted as a request to end the 'Teletel' videotext service session instead of a request to end the 'remote' videotext service session.

Workaround : 'remote' videotext server may request ASAP an automatic reply from the terminal (IE : ENQ ROM) or the user may press any key between <service_code>[Send] and [local/Line] function key.

Bug #3 (known) : "Restart Teletel server"

If initiated from the Display console, all messages from the Teletel server instance will appear in PyMoIP Gateway's terminal and not in a new one.

Workaround : Restart both PyMoIP gateway and the Teletel server instance by a proper script (PyMoIP gateway first, followed by a small delay, then, the Teletel server instance)

Bug #4 (To be investigated) : "Restart PyMoIP Gateway while Teletel server instance is already running"

While it's designed to work correctly (retries and so on), communication between both programs appears to fail time to time

Workaround : Restart both PyMoIP gateway and the Teletel server instance by a proper script (PyMoIP gateway first, followed by a small delay, then, the Teletel server instance)

3. Improvements & functionnalities planned :

  • Redirection to Telnet Videotext servers
  • Simulate 1200bps (or whatever) throutput [implemented from 14/09/2021 - also fixes some 'end of redirection' buffering issues] Needs more parameterisation to be defined

4. Parameter files

4.1 GatewayConfig

It's a simple JSON file named 'GatewayConfig.json' that might be located in PyMoIP dump, logs & config folder (IE : '/home/pi/PyMoIP_dir' by exemple) [but defaults to './Gateway/']

"GatewayExecCount": 0

  • Mandatory parameter
  • If empty/unfoundable .json file, will start count at zero
  • Read and incremented at each Gateway launch
  • Purpose : Count Gateway launches in order to keep dumped data files with unique file names

"DumpData": false

  • Optionnal parameter
  • Defaults to false, may be set to true
  • Read at each Gateway launch
  • Purpose : Activate the dump of all user inputs to keep full user trace (dumped data files in "DumpDataPath") [currently, only for Telnet users]

"DumpDataPath": "~/PyMoIP_dir/"

  • Mandatory parameter when "DumpData"==true
  • Defaults to "~/PyMoIP_dir/Dumps/"
  • Read at each Gateway launch

"LimitTcpSpeed": 1.2

  • Optionnal parameter
  • Defaults to 0, may be set to a float value (1.2 for 1200 bps, 9.6 for 9600 bps)
  • Read at each Gateway launch
  • Purpose : Limit output videotext speed. Also, permit the cancellation of output buffers when the user presses Local/Line to end a redirected session

"LimitWsSpeed": 1.2

  • Optionnal parameter
  • Defaults to 0, may be set to a float value (1.2 for 1200 bps, 9.6 for 9600 bps)
  • Read at each Gateway launch
  • Purpose : Limit output videotext speed. Also, permit the cancellation of output buffers when the user presses Local/Line to end a redirected session

4.2 NeverBan

It's a simple optional JSON file named 'neverbanned.json' that might be located in PyMoIP dump, logs & config folder (IE : '/home/pi/PyMoIP_dir' by exemple) [but defaults to './Gateway/']

Contains a list of @IP that will never be banned, such as :

{"192.168.168.1": "My router" , "199.199.199.199": "Remote friend of mine"}

5. Glossary

Default videotext server : Refers mostly to 'Teletel server instance'. This is the WebSocket videotext server that PyMoIP Gateway will contact and try to connect to the arriving user. It's defined by parameters in the command line.

Display server : For administration usage.

  • Show the trace of starting/ending sessions in real time
  • Show the definition of timeouts
  • Show the current usage status
  • Show available command (h)

Dump, logs & config folder : Refers to a folder that keeps configuration, dump and logs files for PyMoIP.

  • This folder must be Read/Write and persistant (not cleared when host reboots)
  • Written data (and data write rate) to this folder should remain low but volume may increase in time
  • PyMoIP Gateway related files in this folder :
    • PyMoIP Gateway banned file : 'banned.json' (Note : This file is copied to '/tmp' folder by start script and copied back from '/tmp' folder by stop script. As a side effect, banned/unbanned @IP aren't kept in case of system crash, system shutdown, or unclean [CTRL+C, kill -9] stop of the gateway)
    • PyMoIP Gateway configuration file : 'GatewayConfig.json' [possibly, or in 'BASEDIR'/Gateway]
    • PyMoIP Gateway log file : 'PyMoIP_gateway.log' (*)
    • PyMoIP Gateway DumpData files : DumpData__.dmp files, created when option "DumpData" is true (*)

Gateway : Refers mostly to 'PyMoIP Gateway' program.

Teletel server instance : Refers mostly to 'Default videotext server'. This WebSocket videotext server is supposed to be running PyMoIP Server and module_teletel.py, have a command channel port openned for direct communication with the Gateway and allow redirections.

(*) (Note : Not designed to go to '/tmp' folder but a risk to corrupt the filesystem exists in case of very bad crash)

Clone this wiki locally