Using xargs
, described in the Linux xargs manual as a tool which builds and execute command lines from standard input, once can exert a significant amount of additional power over any other command executed on the Bash command line. Basically, xargs
will take the output from any other tool, and use that as it’s own input for further processing and action (hence the reference to executing command lines in the manual). If this is your first few weeks or months with xargs
, or you are only just starting, this is the best place for you to get into xargs
.
In this tutorial you will learn:
- How to use
xargs
from the command line in Bash - How
xargs
works, what it does, and how to use it well - Basic usage examples using
xargs
from the command line in Bash
Software requirements and conventions used
Category | Requirements, Conventions or Software Version Used |
---|---|
System | Linux Distribution-independent |
Software | Bash command line, Linux based system |
Other | The xargs utility is included in the Bash shell by default |
Conventions | # – requires given linux-commands to be executed with root privileges either directly as a root user or by use of sudo command$ – requires givenlinux-commands to be executed as a regular non-privileged user |
Example 1: A simple start
Let’s dive right in with a simple xargs
example:
$ echo '1' > 1 $ echo '2' > 2 $ echo '3' > 3 $ ls 1 2 3 $ ls | xargs cat 1 2 3
In this example, we quickly created 3 files by echoing a number and then redirecting the output (using >
) to 3 individual files named 1
to 3
. After this, we checked the existence of the files with the ls
command (list directory contents).
In the last command, we used the ls
and piped (using |
) it’s output to xargs
. After the xargs
command we listed the new command which we wanted xargs
to execute for any input it received, individually per input, in this case cat
(which outputs the content of a file).
Basically the command above is analogous with the following set of commands (though if the directory contents would change, it would no longer be analogous as the xargs
will simply include new directory entries):
$ cat 1 1 $ cat 2 2 $ cat 3 3
Example 2: A little more complex
$ ls --color=never | xargs -I{} cat {} 1 2 3
Here we added a few Bash syntax idioms (an idiom is a language construct/expression) to vastly improve the quality and perhaps readability of the same one-liner script, though the output – in this case – has remained the same.
The -I{}
option to xargs
defines a replace string which will be used inside the xargs command to insert, at the point where the replace string is used again, the input received, and that for each input individually – i.e. the command specified after the xargs
command will be executed for each input individually.
The --color=never
addition makes the output generated by the ls
command (and therefore the input to the xargs
command) much safer.
Example 3: Buggy output
In the last example we introduced --color=never
as an advisable option. Let’s look at an example of how – without this option – we can run into issues:
$ mkdir 1 2 3 $ ls 1 2 3 $ ls | xargs ls ls: cannot access ''
Interesting output 🙂 So what happened here?
We first created three directories, 1
to 3
, and subsequently checked the creation thereof. Notice the output is dark blue, courtesy of Bash shell colors, which are setup by default on for example Ubuntu.
Next we took the output of ls
and passed it to xargs
which in turn passed it back to ls
. Similar to ‘echo 1 | ls` one would expect. But it did not work! Instead, we received a set of color codes backr. Anyone seeing this for the first time would have a hard time linking it back to color codes.
Adding a simple option --color=never
sanitized the ls
output as more suitable input for xargs
, and the output looks as expected: for the directories 1
to 3
, as output by the ls
command, list the contents (i.e. similar to ls 1; ls 2; ls3
).
Example 4: Text parsing with xargs and sed
$ echo -e '1\n2\n3' > test $ cat test 1 2 3 $ ls test $ ls --color=never | xargs -I{} cat {} | sed 's|[2-3]|0|' 1 0 0 $ ls --color=never | xargs -I{} cat {} | sed 's|[2-3]|0|' | xargs -I{} grep {} $(ls) 1
A bit more challenging example, but you will soon understand it.
The first command simply created a file with three lines (\n
creates a newline, and the -e
option to echo allows the \n
to be used). Next, we output the contents of the file using cat
, and checked how many files were present using ls
(only one; the test
file we created).
Next we have our juicy xargs command, which – as you know from previous examples – takes the directory listing, sends it to xargs
and outputs the contents of each file passed using cat
. Then we have a small sed
command which substitutes the numbers 2
and 3
to 0. The output is the full contents of the test file, but with the 2
and 3
changed to 0
as instructed.
Finally, we take the same command and pass it to another xargs
. Think about it like this; visually (in your mind) see the 1
, 0
, 0
output from the previous command being piped (|
) into the final xargs
without the preceding complexity. See how you can build very complex commands gradually like this? This also shows the power of xargs
in a nutshell, though much more is possible.
In the final command we pass that 1
, 0
, 0
output into grep
, which will search the contents of $(ls)
. $()
starts a subshell, executes whatever command is in it, and inserts the output of that command in the place where the subshell was called. In other words, we are executing:
grep '1' ./test grep '0' ./test grep '0' ./test
And the results is indeed as expected; 1
only, as there were no zeros in the file to start with.
Conclusion
In this article we have looked at the basics of the xargs
command and a few example of how xargs
can be used including text parsing and directory listing manipulation. My next article will be on more advanced xargs
usage, including additional examples.
Enjoy xargs
and leave us a comment with your coolest xargs
command lines or scripts!