xargs for beginners with examples

Using xargs, described in the Linux xargs manual as a tool which builds and execute command lines from standard input, once can exert a significant amount of additional power over any other command executed on the Bash command line. Basically, xargs will take the output from any other tool, and use that as it’s own input for further processing and action (hence the reference to executing command lines in the manual). If this is your first few weeks or months with xargs, or you are only just starting, this is the best place for you to get into xargs.

In this tutorial you will learn:

  • How to use xargs from the command line in Bash
  • How xargs works, what it does, and how to use it well
  • Basic usage examples using xargs from the command line in Bash

xargs for beginners with examples

xargs for beginners with examples

Software requirements and conventions used

Software Requirements and Linux Command Line Conventions
Category Requirements, Conventions or Software Version Used
System Linux Distribution-independent
Software Bash command line, Linux based system
Other The xargs utility is included in the Bash shell by default
Conventions # – requires given linux-commands to be executed with root privileges either directly as a root user or by use of sudo command
$ – requires givenlinux-commands to be executed as a regular non-privileged user


Example 1: A simple start

Let’s dive right in with a simple xargs example:

$ echo '1' > 1
$ echo '2' > 2
$ echo '3' > 3
$ ls
1  2  3
$ ls | xargs cat
1
2
3

In this example, we quickly created 3 files by echoing a number and then redirecting the output (using >) to 3 individual files named 1 to 3. After this, we checked the existence of the files with the ls command (list directory contents).

In the last command, we used the ls and piped (using |) it’s output to xargs. After the xargs command we listed the new command which we wanted xargs to execute for any input it received, individually per input, in this case cat (which outputs the content of a file).

Basically the command above is analogous with the following set of commands (though if the directory contents would change, it would no longer be analogous as the xargs will simply include new directory entries):

$ cat 1
1
$ cat 2
2
$ cat 3
3

Example 2: A little more complex

$ ls --color=never | xargs -I{} cat {}
1
2
3

Here we added a few Bash syntax idioms (an idiom is a language construct/expression) to vastly improve the quality and perhaps readability of the same one-liner script, though the output – in this case – has remained the same.

The -I{} option to xargs defines a replace string which will be used inside the xargs command to insert, at the point where the replace string is used again, the input received, and that for each input individually – i.e. the command specified after the xargs command will be executed for each input individually.

The --color=never addition makes the output generated by the ls command (and therefore the input to the xargs command) much safer.

Example 3: Buggy output

In the last example we introduced --color=never as an advisable option. Let’s look at an example of how – without this option – we can run into issues:

tained ls output after using xargs

ls output is tainted after xargs was used, requiring the use of the –color=never option to ls
$ mkdir 1 2 3
$ ls
1  2  3
$ ls | xargs ls
ls: cannot access ''


Interesting output 🙂 So what happened here?

We first created three directories, 1 to 3, and subsequently checked the creation thereof. Notice the output is dark blue, courtesy of Bash shell colors, which are setup by default on for example Ubuntu.

Next we took the output of ls and passed it to xargs which in turn passed it back to ls. Similar to ‘echo 1 | ls` one would expect. But it did not work! Instead, we received a set of color codes backr. Anyone seeing this for the first time would have a hard time linking it back to color codes.

Adding a simple option --color=never sanitized the ls output as more suitable input for xargs, and the output looks as expected: for the directories 1 to 3, as output by the ls command, list the contents (i.e. similar to ls 1; ls 2; ls3).

Example 4: Text parsing with xargs and sed

$ echo -e '1\n2\n3' > test
$ cat test
1
2
3
$ ls
test
$ ls --color=never | xargs -I{} cat {} | sed 's|[2-3]|0|'
1
0
0
$ ls --color=never | xargs -I{} cat {} | sed 's|[2-3]|0|' | xargs -I{} grep {} $(ls)
1

A bit more challenging example, but you will soon understand it.

The first command simply created a file with three lines (\n creates a newline, and the -e option to echo allows the \n to be used). Next, we output the contents of the file using cat, and checked how many files were present using ls (only one; the test file we created).

Next we have our juicy xargs command, which – as you know from previous examples – takes the directory listing, sends it to xargs and outputs the contents of each file passed using cat. Then we have a small sed command which substitutes the numbers 2 and 3 to 0. The output is the full contents of the test file, but with the 2 and 3 changed to 0 as instructed.

Finally, we take the same command and pass it to another xargs. Think about it like this; visually (in your mind) see the 1, 0, 0 output from the previous command being piped (|) into the final xargs without the preceding complexity. See how you can build very complex commands gradually like this? This also shows the power of xargs in a nutshell, though much more is possible.

In the final command we pass that 1, 0, 0 output into grep, which will search the contents of $(ls). $() starts a subshell, executes whatever command is in it, and inserts the output of that command in the place where the subshell was called. In other words, we are executing:

grep '1' ./test
grep '0' ./test
grep '0' ./test

And the results is indeed as expected; 1 only, as there were no zeros in the file to start with.

Conclusion

In this article we have looked at the basics of the xargs command and a few example of how xargs can be used including text parsing and directory listing manipulation. My next article will be on more advanced xargs usage, including additional examples.

Enjoy xargs and leave us a comment with your coolest xargs command lines or scripts!



Comments and Discussions
Linux Forum