Site Map - skip to main content

Hacker Public Radio

Your ideas, projects, opinions - podcasted.

New episodes every weekday Monday through Friday.
This page was generated by The HPR Robot at


hpr2793 :: bash coproc: the future (2009) is here

clacke discovers bash's coproc keyword and explains some toy examples

<< First, < Previous, , Latest >>

Thumbnail of clacke
Hosted by clacke on 2019-04-17 is flagged as Clean and is released under a CC-BY-SA license.
bash, coproc, subshell. 5.
The show is available on the Internet Archive at: https://archive.org/details/hpr2793

Listen in ogg, spx, or mp3 format. Play now:

Duration: 00:21:16

Bash Scripting.

This is an open series in which Hacker Public Radio Listeners can share their Bash scripting knowledge and experience with the community. General programming topics and Bash commands are explored along with some tutorials for the complete novice.

If you want the full manuscript, that’s at gitlab: hpr2793_bash_coproc_manuscript.adoc. It’s almost a transcript, but I added spontaneous commentary while reading the examples, so that’s not in the manuscript.

Episode errata:

  • Command substitution with $() is perfectly valid according to POSIX, and is accepted both by dash and by bash --posix. It’s not to be considered a bashism.

  • I fumbled the pronunciation of the printf format string in one place and said "parenthesis" instead of "percentage sign".

  • I tried to say "space" every time there’s a space, but I know I forgot it in a few places. But you probably need to look at the show notes to really make sense of the commands anyway.

Example #1:

More on command substitution in Dave’s hpr1903: Some further Bash tips.

Example #2:

You can also combine process substitution with redirection.

Example #3:

More on process substitution in Dave’s hpr2045: Some other Bash tips.

For a description of a hack for creating bidirectional anonymous pipes in bash, see my Fediverse post on this, and I owe you a show.

A coprocess in bash is a subshell to which you have access to two file descriptors: Its stdin and its stdout.

The two file descriptors will be put in a bash array. To learn more about arrays, check out Dave’s series within the bash series, a whopping five-part quadrology including hpr2709, hpr2719, hpr2729, hpr2739 and hpr2756.

You create a coprocess using the coproc keyword, brand spanking new since bash 4 from 2009. I am filing issues to pygments and GNU src-highlite to support it.

There are two ways to call coproc. The first way is to give coproc a simple command.

Example #4:

The other way is to give coproc an explicit name and a Command Grouping.

Example #5:

Slightly less contrived example #6:

$ coproc GREP (grep --line-buffered pub); printf '%s\n' hacker public radio >&${GREP[1]}; cat <&${GREP[0]}
[1] 25627
public
^C
$ kill %1
[1]+  Terminated              coproc GREP ( grep --color=auto --line-buffered pub )

Here grep and cat wait forever for more input, so we have to kill them to continue our lesson.

But we know that GREP will only return one line, so we can just read that one line. And when we are done feeding it lines, we can close our side of its stdin, and it will notice this and exit gracefully.

I’m glad I stumbled over that {YOURVARIABLE}>&- syntax for having a dereferenced variable as the left FD of a redirection. Originally I used an ugly eval.

Example #7:

$ coproc GREP (grep --line-buffered pub); printf '%s\n' hacker public radio >&${GREP[1]}; head -n1 <&${GREP[0]}; exec {GREP[1]}>&-
[1] 25706
public
[1]+  Done                    coproc GREP ( grep --color=auto --line-buffered pub )

There we go! Not the most brilliant example, but it shows all the relevant moving parts, and we covered a couple of caveats.

Now go out and play with this and come back with an example on how this is actually useful in the real world, and submit a show!


Comments

Subscribe to the comments RSS feed.

Comment #1 posted on 2019-04-22 12:09:22 by Dave Morriss

I really enjoyed this!

Thanks clacke, I enjoyed this a lot. Nice to be on the receiving end of the Bash info for a change ;-)

Command substitution: It's my understanding that the newer $() form is an improvement on the older `` form largely because the substitutions can be nested. At least, when I found it years ago I was excited to be able to nest them! I assume it's nestable because the new form is easier to parse.

coproc: This seems cool, though a little involved. I'd looked briefly but hadn't really thought about the feature. Thanks for covering it.

Since b-yeezi and I have awoken the Awk series from its hibernation recently, I'm going to cover redirection and Awk's coprocess feature as well. Episode 15 is almost ready to be recorded and uploaded.

Comment #2 posted on 2019-05-04 11:20:57 by clacke

Re: backquotes vs dollar-paren

Yes, that's why the dollar-paren was introduced. Backquotes can be nested too, but that requires escaping them with backslashes and we don't want to go there if we can avoid it.

Apart from the nesting thing, I find dollar-paren easier to read, especially when enclosed in double-quotes, as it usually is.

I thought that one difference between the two is that dollar-paren trims any trailing newlines, but it turns out I was wrong -- they both do that, so the difference is purely about quoting and readability.

Comment #3 posted on 2019-05-09 04:17:22 by clacke

Re: awk coprocesses

I heard your comment again on the Community News and discovered that you were talking about awk coprocesses also, not just backticks. :-)

I didn't know about them! Looking forward to the episode.

For anyone wanting to read ahead of the class, the documentation page is here:

https://www.gnu.org/software/gawk/manual/html_node/Two_002dway-I_002fO.html

Comment #4 posted on 2019-05-09 08:57:35 by Dave Morriss

Regarding awk coprocesses

Hi clacke,

Yes, thanks for the link. I thought "I put that link in my notes", went and looked and found I'd messed it up, so I just fixed it :-)

The show is number 15 in the series and is now on the site as https://hackerpublicradio.org/eps.php?id=2824

Comment #5 posted on 2021-10-08 10:58:59 by clacke

Real world use, thanks Dave!

"Now go out and play with this and come back with an example on how this is actually useful in the real world, and submit a show!"

Dave Morriss did so in HPR3413:

https://hackerpublicradio.org/eps.php?id=3413

Even though he concludes that this feature is awkward compared to the alternatives, it's great to have a less contrived example!

Leave Comment

Note to Verbose Commenters
If you can't fit everything you want to say in the comment below then you really should record a response show instead.

Note to Spammers
All comments are moderated. All links are checked by humans. We strip out all html. Feel free to record a show about yourself, or your industry, or any other topic we may find interesting. We also check shows for spam :).

Provide feedback
Your Name/Handle:
Title:
Comment:
Anti Spam Question: What does the letter P in HPR stand for?
Are you a spammer?
What is the HOST_ID for the host of this show?
What does HPR mean to you?