Skip to content

Commit

Permalink
adds python implementations
Browse files Browse the repository at this point in the history
- more README
- fix some C++ code along the way
  • Loading branch information
nojhan committed Aug 30, 2022
1 parent 77db20f commit 7ceee60
Show file tree
Hide file tree
Showing 5 changed files with 85 additions and 43 deletions.
76 changes: 61 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,32 +1,61 @@
Named pipes services
====================

Examples of how to design services that use Linux' named pipes FIFO as I/O.
Examples (in C++ and Python) of how to design services that use named pipes FIFO as I/O.

Instead of implementing heavy web services or complex low-level network code,
just read/write from/to files, and be done with it.

Rationale
---------

Introduction
------------

### Rationale

The problem of making two programs *communicate* is among the ones
that generated the largest litterature and code in all computer science
(along with cache invalidation, naming things, and web frameworks).

When facing such a problem, a programer immediatly thinks "I'll use middleware".
If you don't really now what a middleware is, be at ease, nobody really knows.
Nowadays, you may have eared of their latest avatar: *web services*.
As our programmer is going to realize, one now have *two* problems.
The burden of writing, using, and maintaining code using middleware is always huge.
Because they are made to handle a **tremendous** number of complex situations,
most of which involve adversary users, users being bots, or both.

But most of the time, the actual problem does not really involve these situations.
At least not at the beginning (which means probably never).
If you are building up (firsts versions of) communicating programs
that will run on a (safe) local network,
and for which the exchanged messages are known,
then I have good news:
**you don't have to use web services** (or any kind of middleware).

**You just need to know how to read/write from/to (special) files**.


### Overview

The basic idea is that, instead of programming the network interface
to your service with low level sockets or any high level library,
you can just implement query/answer mechanisms using named pipes.
you can just implement query/answer mechanisms using **named pipes**.

Named pipes are special FIFO files that are blocking on I/O
Named pipes are special *FIFO* files that are blocking on I/O
and implements a very basic form of message passing,
without having to bother with polling.
Moreover, they are very easy to use, are they are just files
Moreover, they are very easy to use, as they are just files
in which you read/write.

Once you made your service on top of named pipes,
it is easy to wrap it within an interface made with other languages/tools.
For instance, it is very easy to expose it on the network using common Linux tools like `socat`.
For instance, it is very easy to expose it on the network using common tools like `socat`.

Be warned that this is not secure, though, you should only use this for testing
purpose in a secure local network.
purpose in a secured local network.


Principle
---------
### Principle

The theoretical principle can be represented by this UML sequence diagram:
```
Expand All @@ -48,24 +77,40 @@ The theoretical principle can be represented by this UML sequence diagram:
│ │ │ │
```

Note that the service is started first and is waiting for the input.
Note also that there are two pipes, here: one for the input and one for the output.
Notes:
- the service is started first and is waiting for the input,
but as processes are blocking, the starting order does not always matter.
- there are two pipes, here (one for the input and one for the output),
for the sake of simplicity, but you may just as well use only one.


Build and run
-------------

Python code does not need to be built.

To build the C++ code on Linux, just call:
```sh
./build.sh
```

You may use the `run_*` scripts to see how to run the examples.
For instance, for the most complex one:
```
./run_service2.sh
```


Examples
--------

To create the named pipes under Linux, use the `mkfifo` command, as shown in the `build.sh`
To create the named pipes under Linux or MacOS, use the `mkfifo` command, as shown in the `build.sh`
script.

Creating named pipes on windows is more complex, you may want to look at the
[related Stack Overflow question](https://stackoverflow.com/questions/3670039/is-it-possible-to-open-a-named-pipe-with-command-line-in-windows)


### Trivial example: a `cat` service

The `pcat` executable implements a service that reads from a named pipe and
Expand Down Expand Up @@ -125,6 +170,8 @@ Use `Ctrl-C` to close the remaining `cat` process.
Furthermore
-----------

### Expose such services on network

If you want to expose such a service as a network server, just use socat.

For example, to get _data_ query from the network for `service1`:
Expand All @@ -151,8 +198,7 @@ socat TCP-LISTEN:8478,reuseaddr,fork PIPE:/./data
```


Troubleshooting
===============
### Troubleshooting

If you witness strange behavior while debugging your own services (like prints
that do not occur in the correct terminal), double check that yo don't have some
Expand Down
13 changes: 8 additions & 5 deletions pcat.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
#!/usr/bin/env python3
#
#!/usr/bin/env python

import sys

while True:
with open(sys.argv[1],'r') as fd:
print(fd.read(), flush=True)
if __name__ == "__main__":

while True:
with open(sys.argv[1]) as fin:
line = fin.readline()
sys.stdout.write(line)

3 changes: 2 additions & 1 deletion run_service1.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@

./service1 data > out &
# ./service1 data > out &
./service1 data out &
PID_SERVICE=$!

echo "Hellow World!" > data &
Expand Down
2 changes: 1 addition & 1 deletion service1.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,9 @@ int main(int argc, char** argv)
ifs.close();

std::string data = strip(datas.str());
std::clog << "Received: <" << data << ">" << std::endl;

std::ofstream ofs(argv[2]);
std::clog << "Received: <" << data << ">" << std::endl;
ofs << data << std::endl;
ofs.close();

Expand Down
34 changes: 13 additions & 21 deletions service2.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,9 @@

#include <sys/stat.h>

enum ERROR { NOT_FIFO=1, NO_CONTEXT };
enum ERROR { NOT_FIFO=1 };

class Service
{
class Service {
protected:
bool _has_current_context;
std::mutex _mutex;
Expand All @@ -20,14 +19,11 @@ class Service
std::string _out;
std::string _current_context;

bool has_current_context()
{
std::lock_guard<std::mutex> guarded_scope(_mutex);
bool has_current_context() const {
return _has_current_context;
}

void has_current_context(bool flag)
{
void has_current_context(bool flag) {
std::lock_guard<std::mutex> guarded_scope(_mutex);
_has_current_context = flag;
}
Expand All @@ -45,16 +41,14 @@ class Service
_out(out)
{}

std::string strip(std::string s)
{
std::string strip(std::string s) const {
s.erase(std::find_if( s.rbegin(), s.rend(),
[](int ch) { return !std::isspace(ch); }
).base(), s.end());
return s;
}

void update_current_context()
{
void update_current_context() {
while(true) {
std::clog << "Wait for context..." << std::endl;
bool has_error = false;
Expand All @@ -74,8 +68,7 @@ class Service
}
}

void handle_data()
{
void handle_data() const {
while(true) {
if(this->has_current_context()) {
std::string data;
Expand Down Expand Up @@ -104,13 +97,12 @@ class Service
out.close();
std::clog << "\tdone" << std::endl;
} // if not has_error
}
} // if has context
} // while true
}
};

bool is_named_pipe_fifo(char* filename)
{
bool is_named_pipe_fifo(char* filename) {
struct stat st;
stat(filename, &st);
if(not S_ISFIFO(st.st_mode) ) {
Expand All @@ -119,11 +111,11 @@ bool is_named_pipe_fifo(char* filename)
return true;
}

int main(int argc, char** argv)
{
assert(argc = 3);
int main(int argc, char** argv) {

assert(argc == 4);

for(size_t i=1; i < 3; ++i) {
for(size_t i=1; i < 4; ++i) {
if( not is_named_pipe_fifo(argv[i]) ) {
std::cerr << "ERROR: " << argv[i] << " is not a named pipe FIFO" << std::endl;
exit(ERROR::NOT_FIFO);
Expand Down

0 comments on commit 7ceee60

Please sign in to comment.