The Joys of xargs

Tuesday Sep 2nd 2008 by Juliet Kemp
Share:

Juliet Kemp explores the incredibly useful xargs command. It takes in input and executes your chosen command on it. Deceptively simple in concept; extremely powerful in execution.

xargs is an incredibly useful command: it takes in input and executes your chosen command on it: xargs command. Deceptively simple in concept; extremely powerful in execution. It's most commonly used to execute one command on the output from another command (command1 | xargs command2), and most often the output-generating command is find (xargs is more flexible than the built-in -exec option). Here we'll look at xargs with find, and then at some other possibilities.

A couple of options which are useful for testing: -t echoes the command before executing it, and -p echoes it and asks for confirmation.

xargs and find

find and xargs do go very well together: find to locate what you're looking for, and xargs to run the same command on each of the things found.

Traditionally, an advantage to xargs was its ability to handle long command lines before failing, unlike some other commands. This command:

rm `find tmp -maxdepth 1 -name '*.mp3'`
is intended to remove all tmp/*.mp3 files (and ignore any subdirectories), but can fail with an "Argument list too long" message. This exact equivalent:
find tmp -maxdepth 1 -name '*.mp3' -maxdepth 1 | xargs rm
does exactly the same thing but will avoid the problem by batching arguments up. More modern kernels (since 2.6.23) shouldn't have this issue, but it's wise to make your scripts as portable as possible; and the xargs version is also easier on the eye.

You can also manually batch arguments if needed, using the -n option.

find tmp -maxdepth 1 -name '*.mp3' -maxdepth 1 | xargs -n1 rm
will pass one argument at a time to rm. This is also useful if you're using the -p option as you can confirm one file at a time rather than all at once.

Filenames containing whitespace can also cause problems; xargs and find can deal with this, using GNU extensions to both to break on the null character rather than on whitespace:

find tmp -maxdepth 1 -name *.mp3 -print0 | xargs -0 rm
You must use these options either on both find and xargs or on neither, or you'll get odd results.

Another common use of xargs with find is to combine it with grep. For example,

find . -name '*.pl' | xargs grep -L '^use strict'
will search all the *.pl files in the current directory and subdirectories, and print the names of any that don't have a line starting with 'use strict'. Enforce good practice in your scripting!

Moving on from find: it can be useful to pipe the contents of a file into xargs as input. So,

xargs -t -n2 diff < diff-files
would take the arguments listed in the file diff-files in groups of 2 and run diff on them. So if the diff-files file consisted of:
sample1 alternate1
sample2 alternate2
then xargs would run:
diff sample1 alternate1
diff sample2 alternate2
This can be a quick way of comparing large numbers of files. (Use -p instead of -t to get a pause after each diff as well as an echo of the command.)

You can also use a listings file and xargs to concatenate the contents of those files:

xargs cat < list-of-files > files-contents
(generate list-of-files using xargs as well!
find . -maxdepth 1 -name '*.tex' | xargs echo > list-of-files
would get all your LaTeX source files in the current directory into one list, ready to be stuck together.)

You can use xargs if you need to rename lots of files (e.g. datestamping). This command will rename each file in the current directory from filename.txt to 20080815-filename.txt:

ls | xargs -I {} mv {} 20080815-{}
This works because {} is a placeholder meaning "the current argument". (You can use xxx or yyy or any other string instead of {} if you want, as well, and it'll do exactly the same thing.) -I implies -n1, because you want to act on each file individually.

Or you might want to move all the files in directory 1 into directory 2:

ls dir1 | xargs -I {} -t mv dir1/{} dir1/{}

Conclusion

I've concentrated here on using xargs to manipulate files in various ways, but you can use the same tricks for other commands. For example, if you have a file containing a list of IP addresses,

cat iplist | xargs -n1 nmap -sV
would run nmap on each IP address at a time. Play around with it a bit and see what you can do!

This article was first published on LinuxPlanet.com.

Please enter your content here.
Share:
Home
Mobile Site | Full Site
Copyright 2017 © QuinStreet Inc. All Rights Reserved