This is Hacker Public Radio Episode 3,811 from Monday the 13th of March 2023.
Today's show is entitled, Make Feeful and Named Pipes.
It is part of the series' bash scripting.
It is hosted by Clottu and is about 11 minutes long.
It carries a clean flag.
The summary is, have you ever named a pipe?
If not, this is the episode you've been waiting for.
Hey everybody, this is Clottu on Hacker Public Radio.
Today I wanted to talk about Make Feeful, that is MKFIFO.
But in order to understand Make Feeful, you have to understand what a Feeful is or a FIFO.
Let's see, if I do it, man, FIFO, FIFO.
First, in first, out special file, commonly called a named pipe.
You may have heard of a named pipe.
It usually appears in a bullet list of file types on Linux or something like that and it's always
sort of buried at the bottom because a lot of people making these lists don't actually
use the thing that they're listing so they're just like, you know, there's a file and
like a directory and then this named pipe, yeah, that thing and they kind of leave it
at that.
And to be honest, that's where it kind of kept, that's where it stayed for me for a long
time.
I didn't really understand what a named pipe was.
Didn't know when I could use a named pipe or why I would use it still, I guess, kind
of don't.
And I think if named pipes were taken away from me forever, I think I would survive.
I don't think I would complain that much.
But there are some minor conveniences that they do provide.
And I guess I'll talk about some of those here.
But if you have ways that you use named pipes that you rely on or that you think are
really cool, I'd love to hear about it in comments because yeah, I'm still looking for
like that sort of definitive use case for a named pipe.
Well, before talking about what you can do with a named pipe and how to do it, I should
talk really just quickly and briefly about what a pipe is.
So if you, if you don't know what a pipe is, then for instance, on Linux or a unix terminal
you should be able, you can, you could say something like echo, hello world.
And that returns the words, hello world to your terminal.
Well, one of the cool things about the terminal is that you can pipe the output of something
like in this case, the phrase, hello world, you can pipe that as it were into another
command.
And so for instance, if I typed, rev, hello world, hit return, that doesn't work.
Didn't know that didn't work.
How about if I do a, none of those things work without a pipe?
Okay.
Well, what a great way to use a pipe in an example, then?
So echo, hello world and then the pipe symbol, which is like that vertical bar, and
then the word rev, for instance, that echoes the word hello world, the phrase, hello
world.
But it, before it gets to you, before it gets to your terminal output, it goes through
this imaginary pipe, this vertical bar on your keyboard.
And it gets processed by this other command, rev, which reverses whatever it gets as it's
input.
So echo, hello world sends that output through the pipe into the input of rev.
And then that, the natural flow of that is to spit out output into your terminal, just
like echo would have if we hadn't put a pipe in front of it.
And so we get the word, we get the phrase, hello world, except spelled backwards, D L R O
W O L L E H.
That's a very simple example.
And there are lots of really cool examples of what you can do with pipes, because again,
it's super power is that it intercepts the output of one command and sends it to another
command.
Not all commands know to look for input from a pipe.
So you might have to do some special arrangement or some special syntax, often with a dash
symbol to tell that command, hey, don't look, don't look in a file for your input.
Look for it from to standard in for your input.
So that's something that happens, but essentially a pipe takes something from standard
out and pipes it or sends it to standard in.
That's what that does.
So knowing that, and that's an unnamed pipe, that's just a normal every day pipe.
It has a symbol on the keyboard, it's usually, at least on the US keyboard, it's above
the backslash key or it is on the backslash key shift backslash gives you a pipe.
All right, now that we all understand what a pipe is, what's a named pipe?
Name to pipe is similar, but it is a, it is a file object to your computer, but it essentially
does the same thing.
It takes input from one rather output from one place and sends it or kind of puts it into
a holding pattern and then sends it to the input of something else on demand.
So for instance, you could do echo, quote, hello world, redirect, so that's the greater
than symbol or the right, right angle bracket, the one that's points to the right.
And then, oh wait, we don't have a named pipe yet.
We first need to make a named pipe, that's McFetho, MKFIFO.
We'll call it my pipe, okay, so now if I do an LS in my current directory, I see that
there's a special file here, or a file here, that if you have, if you do an LS dash,
what does a capital F, I just have it turned on all the time.
I think it's capital F for classify, it'll show you that this file is not like the other
files.
It's a file name, in this case my pipe, and then the pipe symbol at the end of it.
So to noting that this is a special file is specifically, well, you can find out what
specifically is.
You do file my pipe, it is a FIFO, FIFO named pipe.
So if I, for instance, cat my pipe, nothing happens, in fact, it steals my prompt.
I can't get out of it, so control C, control C, all right, we're back.
So what can we do with this?
Well, you can do an echo, hello world, redirect, that's the right angle, bracket, into
my pipe, and once again, it kind of steals your prompt, so it just kind of hangs there.
Okay, well, that's fine, open up another terminal window and do a cat of my pipe, wherever
my pipe happens to be, and obviously you need to know the location of that, which I was
honestly sure that I did know, there it is, okay.
So cat, till the path to my pipe, I get hello world in this new terminal, spit out into
the output, I go back to the original terminal, and I'm just back at a prompt.
So I have managed to echo hello world into another terminal on my system, that shouldn't
be possible.
Echo hello world just sends a thing to standard output in this terminal, it shouldn't
be able to appear in another terminal, and yet that's what my pipe has made possible.
So that's what my, a named pipe does, it takes your, your data from standard in and holds
it, and it kind of just holds it there until something comes by and opens up that release
valve and gets the output at whatever location there in.
It's kind of cool, kind of fun.
I mean, like I say, it is a little bit weird to a little bit spectacular to be able to
dump output into a file in one location, so here's an LS BLK redirect my pipe, and then again
cat path to my pipe in a different terminal, and I get the output of LS BLK in this
other terminal.
So that's kind of a neat trick on its own.
The other use case that I could think of was sort of as a replacement of temporary files,
granted you're still making a file, but you're only making potentially one file.
So for instance, let's, let's imagine a really test, a really simple test script here.
So hash bang slash bin slash SH echo quote hash tag, or whatever they're called, space
hello world close quote right angle bracket for redirection my pipe, and then the
ampersand symbol, so that we get, we don't get caught in our pipe, and then let's do a
pan doc space dash dash from mark down space to dash dash to HTML space dash dash output
test dot HTML left angle bracket my pipe, okay.
So we're putting the word, we're putting the phrase hello world, pre-pended with a
octo-thorp into my pipe. So now it's, it's in my pipe, it's in a holding pattern.
We use the ampersand symbol to sort of get out of the pipe, and then we use pan doc,
but we for the input of of pan doc, we are redirecting the contents of my pipe in the
pan doc. So if I run that script, SH space dot slash test dot SH, and then cat test
dot HTML, I get H1 ID equals hello dash world, closed quote hello world, closed tag.
So pan doc has successfully run and converted the text hello world to a heading in HTML,
but there has not been, where did that HTML come from? There was never a, there was never a
markdown file for pan doc to process. Well, there was there was the named pipe, and it contained
all that data, the one line of data, and then pan doc was able to act on those contents.
That's a simple example, and any one off example isn't going to seem probably all that useful,
but I could imagine a named pipe being a useful thing to use during a longer process.
So for instance, maybe you have, you know, some, some parsing that you do every day or multiple
times a day. Well, rather than creating a temporary file every single time that cron job kicks off,
just make one named pipe and use that location as sort of your temporary data storage for the
process that runs all the time anyway. That's not to say that my pipe is super intelligent, like
if you try to do something like echo hello world, redirect into my pipe, I guess we'll do an
ampersand just to get our prompt back, and then do for instance a cat of my pipe, which normally
would print hello world. So then I'll pipe that to rev that works as expected. So now I'm going to
echo hello world into my pipe again, and now I'm going to do a cat of my pipe pipe that into rev,
and then redirect the output of that back into my pipe. And now I'm trapped. I don't know where I am.
Well, I'm at a prompt, but I am not able to, like I'm not getting anything out of my pipe,
or I am, but I'm getting nothing out of my pipe. So don't try to take something out of the
name to pipe and then put it right back into it, that doesn't work that way. I mean, you can
reuse the the named pipe, but you just, you cannot, you can't literally like cat the contents of
my pipe to your to it to output, which you then pipe back into the input of my pipe, that it does
not do what you would think it would do. Okay, I think that's everything I have to say about
named pipes. If you have something that you do with named pipes, record an episode about it. I'd
love to hear about what you're doing with them, and that's it. Thanks for listening.
Thank you for watching, and I'll see you in the next video.