A simple implementation of UNIX |
, <
, <<
, >
and >>
in pure C.
In this projects we implement UNIX pipes and redirections, namely
|
, <
, <<
, >
and >>
.
These mechanisms allow us to link commands and files together,
where the output of one serves as the input of the next.
This toolbox philosophy is one of the reasons why UNIX is still popular after 40 years: it encourages developers to create programs that do one thing well, and that work well with other programs. It's also extremely modular and versatile.
I learned about child processes, filesystem interfaces
and the internals of bash
, the target of our emulation.
Since I did both bonuses I'm pretty confident about
the next big challenge in the 42 curriculum: minishell
.
It's basically this with a parser and dynamic environment variables.
- DONT TURN IN LIBS AS SUBMODULES
- MAKEFILE EXPLICIT SOURCE FILES (
echo M_SOURCES
) - Make must compile without relinking
- SHOOULDNT RECOMPILE/REARCHIVE OBJECTS WITH MAKE ALL
- Add
.keep
to object dirs - Remove
initialize
fromall
/build - Create non-phony rule for each lib archive
- Add
- SHOOULDNT RECOMPILE/REARCHIVE OBJECTS WITH MAKE ALL
-
.linux
file (42 Workspaces) - Test in workspaces
- Follows
norminette 3.3.51
- Turn in
Makefile
,*.h
,*.c
,.linux
,.gitignore
- Makefile rules:
$(NAME)
all
clean
fclean
re
- Program name
pipex
- Compiles with
-Wall -Wextra -Werror
- Should not quit unexpectedly (segmentation fault, bus error, double free, etc.)
- All allocated heap memory properly freed, no memory leaks.
- Check memory leaks with
valgrind
- Check memory leaks with
- Allowed functions:
-
open
,close
,read
,write
,malloc
,free
,perror
,strerror
,access
,dup
,dup2
,execve
,exit
,fork
,pipe
,unlink
,wait
,waitpid
-
libft
allowed - Your
ft_printf
(may be modified)- No
printf
fromstdio.h
- No
-
- Handle arguments
file1 cmd1 cmd2 file2
- Wrong number of arguments exits with help message
- Behaves exactly like
< file1 cmd1 | cmd2 > file2
< file1 cmd1
:< file1
readfile1
and pipes it tocmd1
’sSTDIN
cmd1 | cmd2
:cmd1
’sSTDOUT
is piped tocmd2
’sSTDIN
cmd2 > file2
:cmd2
’sSTDOUT
is piped tofile2
, overwriting the file
- Handle errors exactly like
< file1 cmd1 | cmd2 > file2
- Handle input errors (in order):
-
argc != 5
, return0
- No
infile
, return0
- No
cmd1
, return0
- No
cmd2
, return127
-
cmd1
bad arguments, return0
-
cmd2
bad arguments, return1
-
- Pass all testers
- Handle multiple pipes:
< file1 cmd1 | cmd2 | cmd3 ... | cmdn > file2
./pipex file1 cmd1 cmd2 cmd3 ... cmdn file2
- Support
«
and»
when the first parameter is "here_doc":./pipex here_doc LIMITER cmd cmd1 file
cmd << LIMITER | cmd1 >> file
Clone the repo and build with make
:
$ git clone --recurse-submodules https://github.com/librity/ft_pipex.git
$ cd ft_pipex
$ make
It works exactly like < infile cmd1 | cmd2 > outfile
:
./pipex infile "ls" "wc" outfile
./pipex infile "ls -l" "wc -l" outfile
./pipex infile "grep a1" "wc -w" outfile
./pipex .gitignore "tr a b" "tr b c" outfile
./pipex EOF "tr a b" "tr b c" outfile
./pipex .gitignore "ping 8.8.8.8" "grep ms" outfile
You can also compile the bonus implementation:
$ make bonus
It handles multiple pipes < infile cmd1 | cmd2 | cmd3 | ... | cmdn > outfile
:
./pipex infile "grep a" "grep d" "grep s" outfile
./pipex infile "ls -l" "grep a" "wc -l" outfile
./pipex infile "cat" "tr a b" "tr b a" "tr a b" "tr b a" outfile
It also takes a heredoc as input cmd1 << LIMITER | cmd2 >> file
:
./pipex here_doc l "grep a" "grep d" "grep s" outfile
./pipex here_doc l "grep a" "wc -l" outfile
./pipex here_doc x "cat" "tr a b" "tr c d" outfile
Redirect infile
’s contents to cmd
’s STDIN
python hello.py < foo.txt # feed foo.txt to stdin for python
diff <(ls -r) <(ls) # Compare two stdout without files
Redirects previous cmd1
's STDOUT
to cmd2
’s STDIN
ls -la | wc -l
Truncates outfile
with cmd
’s STDOUT
python hello.py > output.txt # stdout to (file)
python hello.py >> output.txt # stdout to (file), append
python hello.py 2> error.log # stderr to (file)
python hello.py 2>&1 # stderr to stdout
python hello.py 2>/dev/null # stderr to (null)
python hello.py &>/dev/null # stdout and stderr to (null)
# GET ALL ERRNO CODES
$ sudo apt install moreutils
$ errno -l
- https://www.man7.org/linux/man-pages/man2/open.2.html
- https://jameshfisher.com/2017/02/24/what-is-mode_t/
Checks if the calling process can access a specific file
Adjust file descriptors so they reference the same file.
- https://www.man7.org/linux/man-pages/man2/dup.2.html
- https://www.geeksforgeeks.org/dup-dup2-linux-system-call/
Replaces the current process with a system call.
- https://jameshfisher.com/2017/02/05/how-do-i-use-execve-in-c/
- https://stackoverflow.com/questions/29615540/using-execve-in-c
- https://www.tutorialspoint.com/unix_system_calls/execve.htm
- https://www.man7.org/linux/man-pages/man2/execve.2.html
- https://stackoverflow.com/questions/5429141/what-happens-to-malloced-memory-after-exec-changes-the-program-image/5429592#5429592
Forks the parent process, creating a child process. The child process is almost identical to its parent, with a copy of its memory space at the time of fork.
- https://www.man7.org/linux/man-pages/man2/fork.2.html
- https://www.youtube.com/watch?v=cex9XrZCU14&list=PLfqABt5AS4FkW5mOn2Tn9ZZLLDwA3kZUY
Creates a pipe: a data channel through which processes can communicate.
Emitting processes can write topipefd[1]
,
and receiving ones can read from pipefd[0]
.
- https://www.man7.org/linux/man-pages/man2/pipe.2.html
- https://man7.org/linux/man-pages/man7/pipe.7.html
- https://www.programacaoprogressiva.net/2014/09/Pipes-em-C-Comunicao-entre-Processos-IPC-Interprocess-Communication.html
Wait for a process to change state.
- https://man7.org/linux/man-pages/man2/wait.2.html
- https://www.man7.org/linux/man-pages/man3/waitpid.3p.html
A string array with all the environment variables of the parent shell.
- https://www.geeksforgeeks.org/c-program-print-environment-variables/
- https://www.geeksforgeeks.org/command-line-arguments-in-c-cpp/
- https://stackoverflow.com/questions/54723587/what-does-envp-stand-for
- Threads are lighter: easier to create and destroy
- Processes don’t share memory
A linked list in the control structure with all the allocated memory pointers.
The interface function ft_lalloc
allocates memory and adds the pointer to the list.
The interface function ft_free_lalloc
frees all pointers and the list.
Part of the larger 42 Network, 42 São Paulo is a software engineering school that offers a healthy alternative to traditional education:
- It doesn't have any teachers and classes.
- Students learn by cooperating and correcting each other's work (peer-to-peer learning).
- Its focus is as much on social skills as it is on technical skills.
- It's completely free to anyone that passes its selection process - The Piscine
It's an amazing school, and I'm grateful for the opportunity.
- https://wikiless.org/wiki/Pipeline_(Unix)?lang=en
- https://wikiless.org/wiki/Unix_philosophy?lang=en
- https://devhints.io/bash
- https://stackoverflow.com/questions/9834086/what-is-a-simple-explanation-for-how-pipes-work-in-bash
- https://stackoverflow.com/questions/21287848/function-to-find-the-maximum-number-of-pipes
- https://linuxconfig.org/introduction-to-named-pipes-on-bash-shell
- https://linuxhint.com/bash_pipe_tutorial/
- https://www.delftstack.com/howto/linux/pipes-in-bash/
- https://devdocs.io/bash/
- https://www.gnu.org/software/bash/manual/html_node/Pipelines.html
- https://www.gnu.org/software/bash/manual/html_node/Lists.html
- https://stackoverflow.com/questions/66511243/bash-pipe-execution-order
- https://unix.stackexchange.com/questions/37508/in-what-order-do-piped-commands-run
- https://linuxhint.com/bash_pipe_tutorial/
- https://cs61.seas.harvard.edu/site/2021/
- https://cs61.seas.harvard.edu/site/2021/ProcessControl/
- https://cs61.seas.harvard.edu/site/2021/Section11/
A more convenient version of execve
.
- https://github.com/denisgodoy/pipex-tester
- https://github.com/vfurmane/pipex-tester
- https://github.com/Yoo0lh/pipex_tester_42
- https://github.com/HEADLIGHTER/pipexfasttest
- https://github.com/gmarcha/pipexMedic
- https://github.com/gsilva-v/PipexTester
- https://github.com/mariadaan/PIPEXaminator
- https://towardsdatascience.com/why-are-there-so-many-tokenization-methods-for-transformers-a340e493b3a8
- https://nlp.stanford.edu/IR-book/html/htmledition/tokenization-1.html
- https://stackoverflow.com/questions/36584062/should-i-free-memory-before-exit
- https://newbedev.com/should-i-free-memory-before-exit
- https://cboard.cprogramming.com/c-programming/75239-freeing-memory-before-exit.html
- http://c-faq.com/malloc/freeb4exit.html
- https://www.linuxquestions.org/questions/programming-9/to-free-or-not-to-free-before-an-exit-458107/
- https://stackoverflow.com/questions/654754/what-really-happens-when-you-dont-free-after-malloc-before-program-termination
- https://stackoverflow.com/questions/2975831/is-leaked-memory-freed-up-when-the-program-exits
git clone --recurse-submodule REMOTE_REPO
git submodule add REMOTE_REPO
git submodule foreach git pull
git submodule update --init --recursive
- https://stackoverflow.com/questions/33714063/how-to-update-submodules-in-git
- https://stackoverflow.com/questions/59271919/how-to-clone-public-submodule-in-github-actions
- https://stackoverflow.com/questions/50254184/git-submodule-and-fetch
- https://www.w3docs.com/snippets/git/how-to-add-a-submodule-in-git.html
- https://stackoverflow.com/questions/1260748/how-do-i-remove-a-submodule#1260982
- https://stackoverflow.com/questions/2006172/git-how-to-reset-a-remote-git-repository-to-remove-all-commits
- https://linuxize.com/post/bash-shebang/
- https://stackoverflow.com/questions/18219262/evaluating-variables-in-a-string
- https://stackoverflow.com/questions/5947742/how-to-change-the-output-color-of-echo-in-linux
- https://stackoverflow.com/questions/1951742/how-can-i-symlink-a-file-in-linux
- https://stackoverflow.com/questions/9452935/unix-create-path-of-folders-and-file