Regular pipes are the method used to "pipe" the output of one program into the input of another; they are created in memory via a system call and do not exist on any filesystem. Named pipes act like regular pipes but are accessed via a file, called a FIFO special file. Unrelated processes can access this file and communicate.
—Robert Love, Linux System Programming, 2nd Edition, Chapter 1, pp. 14-15.
Pipes, FIFOs, and sockets provide efficient ways to do file-like input and output between processes. Their biggest benefit is the ability to treat the endpoints as formatted input and output streams, just like you would read and write from files, or from streams like standard input and output.
In this studio, you will:
pipe() system call
mkfifo() function
Especially if you've not done so already, in addition to reading
LPI pp. 515-522 and 563-571 and LKD pp. 29-33 and 38-40,
please run the following commands on the Linux command line (on your Raspberry Pi
or on one of the Linux Lab machines) for more documentation on what the
fork and exec library functions do:
man 2 fork
man 3 exec
Later in the semester we will discuss Linux processes in greater depth, though if you'd like to do so now you can skim the slides and studio exercises to learn more about parent and child processes.
Please complete the required exercises below, as well as any optional enrichment exercises that you wish to complete. We encourage you to please work in groups of 2 or 3 people on each studio (and the groups are allowed to change from studio to studio) though if you would prefer to complete any studio by yourself that is allowed.
As you work through these exercises, please record your answers, and when you finish them each and every person who worked on them should please log into Canvas, select this course in this semester, and then upload a file containing them and also upload any other files the assignment asks for, and submit those files for this studio assignment (there should be a separate submission from each person who worked together on them).
Make sure that the name of each person who worked on these exercises is listed in the first answer, and make sure you number each of your responses so it is easy to match your responses with each exercise.
pipe() system call to create a
program with communicating child and parent processes. Once created, pipes
cannot be shared between unrelated processes. Thus, the standard way to use
this system call is first to call pipe(), which will
create separate pipe endpoints for reading and writing, and then to call
fork(), which will create a child process. Thus, the child will
inherit the pipe endpoints from the parent, and both processes now will have
access to the pipe.
On your Raspberry Pi, create a C program that:
pipe() to create a pipe with read and write file descriptors (see
man 2 pipe for details)
fork() to create a child process
close(), and the parent process should close its copy of the write
file descriptor similarly
Pipe I/O, like file I/O in C, is comparatively low level. There are two
approaches you can use to read from and write to a pipe. You could use
the integer file descriptors returned by pipe() directly
with the read() and write() system calls, which are
documented at man 2 read and man 2 write.
However you might find it easier to instantiate a FILE* stream variable,
which allows you to work with your pipe like it was any other file object (i.e. you
write to the pipe with fprintf() and read from the pipe with input
operations such as fgets() or fscanf(). You can open a
file stream from a file descriptor with the fdopen() function. You can
Read more about the difference
between file descriptors and file streams here. Of course, all of the above
functions have informative man pages as well.
Hint: Blocking functions that read the pipe will block indefinitely until
they are able to return successfully or the write-end of the pipe is closed by
the writer. The writer should write all of it's data and then call
close() on its file descriptor when it is done. This allows you to
write code such as while( fgets() != NULL ) to do open-ended reads,
on the pipe, which will eventually fail and return once the write-end is closed.
Build and run your program on your Raspberry Pi. As the answer to this exercise, copy and paste the terminal output that is produced by running your program.
PIPE_BUF sized data).
To begin, create a new file for your program code. This program should:
mkfifo(), giving it both read and write permissions (S_IRUSR | S_IWUSR per the table on page 295 of the
LPI textbook). This is documented
at man 3 mkfifo. Be careful not to confuse this function with
the program
called mkfifo that is documented at man 1 mkfifo (which is also the man page you get if you say just man mkfifo).
Note: if the FIFO already exists, the call to mkfifo() will
fail and errno (declared in errno.h, which you can find in the /usr/include/ directory on the machine on which you are compiling in case you would like to browse through it and the other files it includes, to see the symbols and functions it declares) will be set to EEXIST - in that case your program should simply continue to the next step.
fopen() (run the command man 3 fopen to see return values, parameters, etc. for that call).
while loop that continually attempts to read from the
FIFO until EOF is read. Whenever something is read, print it to the standard output.
Build and run your program: it will create a FIFO that will appear in the
filesystem. In a separate terminal, write some data into the FIFO and validate
that it is read by your program correctly. For example, if your FIFO is named
my_ao_fifo, the command echo "my_message" > my_ao_fifo
will insert the string "my_message" into the FIFO. Test your program with larger
amounts of input as well, for example: cat my_file.c > my_ao_fifo.
Now, we will make two modifications to your active object program.
After opening a read stream to the FIFO, also open a write stream
(i.e. fopen( fifo_name, "w" ). Your active object will not ever
write to this stream, but the fact that it is held open will prevent your
program from automatically quitting.
Next, modify the active object to read from the FIFO with the function
fscanf(). This allows you to perform formatted input. Your program
should read integer inputs from the pipe, double the input, and then print out the
original and the new values. Your input stream should not have any non-integer characters
and you do not need to handle this case: test your program by passing integers into your FIFO.
Finally for this exercise, we want to write a set of programs that concurrently use the FIFO to request that the active object perform work. Write two programs that
open the write end of the FIFO with fopen(fifo_name, "w") and
insert integer values with fprintf(). One of these programs should only insert even numbers,
and the other should only insert odd numbers. Run these programs
long enough to verify that they can both write concurrently to the FIFO without
problem. Examine the output of your active object to verify correctness.
As the answer to this exercise, please explain briefly (and show output fragments as evidence) why your program is working correctly based on that output.
man 2 bind under the section
labeled EXAMPLE.
Your server should perform the following actions:
socket() system call. To
create a local connection use the domain AF_LOCAL
(or equivalently AF_UNIX), the connection type
SOCK_STREAM, and protocol zero.
bind() system call. This associates your socket from step
one with a machine-visible address. In the case of
AF_LOCAL, this address is a path in the file system, and
your local server program should use the first argument passed to it
as a string containing that path. Directions on how to specify a local
address in this way (and which header files you'll need in order to make
that work) can be found at man 7 unix, and there
is a good example showing how to do this on page 1166 of the LPI
(Kerrisk) text book.
listen() system
call. Sockets only allow one process to connect at a time, so the second parameter
determines how many connection requests can wait (queue up) before subsequent
ones are rejected.
accept() system call to accept a connection
over the socket interface, passing in NULL and NULL for the second and third arguments. If no connection is immediately available then this
system call will, by default, cause your program to block until another program
attempts to connect. When your program successfully returns from the call to accept() the new communication socket connection has been established, and the return value from the call is a file descriptor for the new socket.
fdopen function and then using fscanf()
as your active object code did to read integers from a FIFO. When
creating the stream, be sure to use the socket descriptor that was
returned by the call to accept() (the communication endpoint),
not the one created by the initial call to socket()
(the connection establishment endpoint). As long as the communication
endpoint socket remains open,
the server should print out all integers that it receives, to its standard
output stream.
unlink() system call in order to destroy
the original local socket (connection establishment endpoint) file before
the program returns.Note: Many things can go wrong with socket-based
communication. As usual, you should always check function
return codes to detect errors. Recall that most functions
return error codes that allow you to print a descriptive error
statement with a line of code such as:
printf("Program error, reason: %s\n", strerror(errno)).
This will require including the headers string.h and
errno.h.
Now you will write a client program that should do the following:
socket()
system call.
connect() system call. This
requires using the same local socket address as was used previously in your
local server program's call
to bind().
Compile your client and server programs, and run them in separate terminal windows on your Raspberry Pi. Copy and paste the client and server programs' output as the answer to this exercise.
accept() (but not to listen() or
bind()).
Your server program should also print a message announcing each time a
new connection is established and should then print out the integer values
it reads from the socket each time, to the standard output stream.
Rather than having your server stay alive forever, now modify your
server program so that it terminates when it recieves a special integer
value (e.g., the http status code
418).
Modify your client so that it can accept zero or one command line arguments, and if it is given the command line argument "quit" it sends that special code to the server after it has sent its other integers (thus
terminating the server).
Compile your refactored server code and run it in its own terminal window. In another window, run your client program several times along with that server to validate that it can stay alive and keep accepting new connections until it is told to quit. Copy and paste the server output as the answer to this exercise.
shell.cec.wustl.edu host.
Note that for this exercise, your server program will run directly on
shell.cec.wustl.edu.
You should not qlogin into a Linux Lab node.
Make copies of your client and server programs, and modify the new copies
to use an internet connection rather than a local connection. Choose a port
number between 30000 and 40000 that both of your programs will use wherever a port
number is needed in the instructions below. Run the command
ping shell.cec.wustl.edu
(on the Raspberry Pi you may need to preface that command with
sudo)
from a terminal window to obtain the numeric IP address of the
shell.cec.wustl.edu machine, which your client
program should use.
netinet/ip.h and arpa/inet.h
in both programs.
socket() (and bind() and connect(), respectively) to specify
an internet connection type with AF_INET
(instead of AF_LOCAL or AF_UNIX). Also remove any
artifacts of the local domain socket from both programs, including removing
the call to unlink() from the server code.
sockaddr_in
(instead of sockaddr_un) to specify the internet
address. This struct is documented at man 7 ip. In both
programs, set the sin_family field to AF_INET, and
the sin_port field to htons(port_num),
where htons() is a function that translates between the
numeric representations used on your computer and in the network, and
port_num is the port number that both your server and
client programs are using. In the server program assign the
sin_addr.s_addr field the value INADDR_ANY, and in
the client program use the function inet_aton()
to translate a c-style string containing the target IP address and
store it in the sin_addr field of the
sockaddr_in struct.
Compile and run your server on shell.cec.wustl.edu.
Compile your client on your Raspberry Pi and run it there twice, once without
telling the server to quit and the second time telling the server to quit.
As the answer to this question, copy and paste the output from your client
and server terminal windows. If you cannot connect for whatever reason,
give a descriptive error message using strerror() and
errno.
Please Note: If your Raspberry Pi is not on the school's network, you will need to first establish a remote connection, either with a VPN connection or through an SSH tunnel.
After you have selected all 3 options, click OK to install the packages (you will be prompted to enter the new password you created on your Pi above).
These instructions have been updated to use the new WashU 2FA (two-factor authentication) based VPN. If you have any problems with enrolling in or using WashU 2FA please call the WashU IT ServiceDesk at (314) 933-3333, or email support@seas.wustl.edu.
Once you have installed the VPN, and have enrolled in WashU 2FA and have chosen which authentication method to use, please open up a terminal window on your Raspberry Pi, and
type:
sudo openconnect danforthvpn.wustl.edu
At the Username: prompt please enter you WUSTL key ID. After that you should get a Password: prompt, where you should enter your WUSTL key password. You will then get a second prompt that also says Password: which is where you should enter one of the following keywords (depending on which authentication method(s) you set up when you enrolled in WashU 2FA):
push -- enter this if you have set up DUO mobile notification
(be aware that if you have more than one device enrolled, you may receive the push on only one device)
phone -- enter this if you want to authenticate via a phone call to the number you have registered with the WashU 2FA system (this may only work if you only have registered a single number there).
(note that other methods like entering sms also may work but involve other details that we have not tested out -
please contact the WashU IT ServiceDesk at (314) 933-3333 or ithelp@wustl.edu, or contact Engineering IT at support@seas.wustl.edu, for help using those other methods.)
Note: you will need to go through this process to start the VPN every time you reboot your Raspberry Pi and want to access the linuxlab machines directly from it.
An SSH tunnel allows you to forward traffic on a specific port to a remote server via SSH.
To establish a tunnel, simply run the following command on your Raspberry Pi:
ssh -f -N your-username@shell.cec.wustl.edu -L port-number:shell.cec.wustl.edu:port-number
You must replace your-username with your WUSTL Key username, and port-number with the port you selected to establish the socket connection.
For example, your command might look like: ssh -f -N msudvarg@shell.cec.wustl.edu -L 31522:shell.cec.wustl.edu:31522
After issuing the command, you will be prompted for your WUSTL Key password.
Alternatively, if you are already set up to log in with an SSH key,
you may replace your-username@shell.cec.wustl.edu with the server nickname,
as detailed on this page.
If you use the SSH Tunnel approach,
your socket code should be configured to instead connect to the local loopback IP address
(i.e. 127.0.0.1) instead of the IP address you obtained for shell.cec.wustl.edu.
Page updated Wednesday, September 23, 2020, by Marion Sudvarg and Chris Gill.