Linux系统中的find命令和xargs命令详解[技巧/资料]

本文最后更新于2014年6月26日,已超过 1 年没有更新,如果文章内容失效,还请反馈给我,谢谢!

在Linux下使用md5sum递归生成整个目录的MD5值
今天要用使用md5sum操作目录,发现它还不支持递归操作,网上有人提出用脚本,其实不用那么麻烦的。
一行命令就搞定了:
find ./ -type f -print0 | xargs -0 md5sum > ./my.md5
检查也很简单:
md5sum -c my.md5
#找到文件夹中比all.log.3新的文件然后将其打包
$ find -depth -newer all.log.3 -type f -exec tar czf log.tgz {} ;#将文件夹中所有的目录打包并压缩

查找最近发生变化的php文件

-type f 表示搜索正常的一般文件   -mtime -7 表示7*24小时内修改的文件

查找文件中是否存在疑似代码

(*代表任意个空格)

注解:很多命令不支持管道传递参数,而实际上又需要这样,所以就用了xargs命令,这个命令可以用来管道传递参数;grep -l表示只包含某个字符串的文件名,如果去掉-l则会显示匹配特定字符串的行内容

比较代码文件

这种情况需要有一份干净的代码,这份代码和正在使用的代码进行比较。例如:

上面的例子是比较wordpress-clean/ 和wordpress-comprised/两个目录,并且目录里面的wp-content/子目录不比较。



  • find

时间参数真是挺有意思的!我们现在知道 atime, ctime 与 mtime 的意义,如果你想要找出一天内被更动过的档案名称, 可以使用上述范例一的作法。但如果我想要找出‘4天内被更动过的档案档名’呢?那可以使用‘ find /var -mtime -4 ’。那如果是‘4天前的那一天’就用‘ find /var -mtime 4 ’。有没有加上‘+, -’差别很大喔!我们可以用简单的图示来说明一下:

图5.2.1、find 相关的时间参数意义
图中最右边为目前的时间,越往左边则代表越早之前的时间轴啦。由图5.2.1我们可以清楚的知道:

  • +4代表大于等于5天前的档名:ex> find /var -mtime +4
  • -4代表小于等于4天内的档案档名:ex> find /var -mtime -4
  • 4则是代表4-5那一天的档案档名:ex> find /var -mtime 4

非常有趣吧!你可以在 /var/ 目录下搜寻一下,感受一下输出档案的差异喔!再来看看其他 find 的用法吧!

上述范例中比较有趣的就属 -perm 这个选项啦!他的重点在找出特殊权限的档案啰! 我们知道 SUID 与 SGID 都可以设定在二进位程式上,假设我想要找出来 /bin, /sbin 这两个目录下, 只要具有 SUID 或 SGID 就列出来该档案,你可以这样做:

因为 SUID 是 4 分,SGID 2 分,总共为 6 分,因此可用 +6000 来处理这个权限! 至于 find 后面可以接多个目录来进行搜寻!另外, find 本来就会搜寻次目录,这个特色也要特别注意喔! 最后,我们再来看一下 find 还有什么特殊功能吧!

find 的特殊功能就是能够进行额外的动作(action)。我们将范例八的例子以图解来说明如下:


图5.2.2、find 相关的额外动作
该范例中特殊的地方有 {} 以及 ; 还有 -exec 这个关键字,这些东西的意义为:

  • {} 代表的是‘由 find 找到的内容’,如上图所示,find 的结果会被放置到 {} 位置中;
  • -exec 一直到 ; 是关键字,代表 find 额外动作的开始 (-exec) 到结束 (;) ,在这中间的就是 find 指令内的额外动作。 在本例中就是‘ ls -l {} ’啰!
  • 因为‘ ; ’在 bash 环境下是有特殊意义的,因此利用反斜线来跳脱。

透过图 5.2.2 你应该就比较容易了解 -exec 到 ; 之间的意义了吧!

如果你要找的档案是具有特殊属性的,例如 SUID 、档案拥有者、档案大小等等, 那么利用 locate 是没有办法达成你的搜寻的!此时 find 就显的很重要啦! 另外,find 还可以利用万用字元来找寻档名呢!举例来说,你想要找出 /etc 底下档名包含 httpd 的档案, 那么你就可以这样做:

不但可以指定搜寻的目录(连同次目录),并且可以利用额外的选项与参数来找到最正确的档名!真是好好用! 不过由于 find 在寻找资料的时后相当的操硬碟!所以没事情不要使用 find 啦!有更棒的指令可以取代呦!那就是上面提到的 whereis 与 locate 啰!


参数代换: xargs

xargs 是在做什么的呢?就以字面上的意义来看, x 是加减乘除的乘号,args 则是 arguments (参数) 的意思,所以说,这个玩意儿就是在产生某个指令的参数的意思! xargs 可以读入 stdin 的资料,并且以空白字元或断行字元作为分辨,将 stdin 的资料分隔成为 arguments 。 因为是以空白字元作为分隔,所以,如果有一些档名或者是其他意义的名词内含有空白字元的时候, xargs 可能就会误判了~他的用法其实也还满简单的!就来看一看先!

其实,在 man xargs 里面就有三四个小范例,您可以自行参考一下内容。 此外, xargs 真的是很好用的一个玩意儿!您真的需要好好的参详参详!会使用 xargs 的原因是, 很多指令其实并不支援管线命令,因此我们可以透过 xargs 来提供该指令引用 standard input 之用!举例来说,我们使用如下的范例来说明:

减号 – 的用途

管线命令在 bash 的连续的处理程序中是相当重要的!另外,在 log file 的分析当中也是相当重要的一环, 所以请特别留意!另外,在管线命令当中,常常会使用到前一个指令的 stdout 作为这次的 stdin , 某些指令需要用到档案名称 (例如 tar) 来进行处理时,该 stdin 与 stdout 可以利用减号 “-” 来替代, 举例来说:

上面这个例子是说:‘我将 /home 里面的档案给他打包,但打包的资料不是纪录到档案,而是传送到 stdout; 经过管线后,将 tar -cvf – /home 传送给后面的 tar -xvf – ’。后面的这个 – 则是取用前一个指令的 stdout, 因此,我们就不需要使用 file 了!这是很常见的例子喔!注意注意!


Linux系统中的find命令和xargs命令详解
参考链接:

  • xargs command is designed to construct argument lists and invoke other utility. xargs reads items from the standard input or pipes, delimited by blanks or newlines, and executes the command one or more times with any initial-arguments followed by items read from standard input. Blank lines on the standard input are ignored.

    xargs is more safer and easy to use

    xargs functionality can be achived using the backquote feature of shell. But, it offers more options. It can deal with blanks or special characters in file names easily. It is often used with find, grep and other commands.

    xargs examples

    For example following example will print 1 2 3 4 using xargs (echo command is default)
    $ echo 1 2 3 4 | xargs echo
    OR
    $ echo 1 2 3 4 | xargs
    You can force xargs to use at most max-args arguments per command line. For example following will use first two argument per command:
    $ echo 1 2 3 4 | xargs -n 2
    Find all .bak files in or below the current directory and delete them.
    $ find . -name "*.bak" -type f -print | xargs /bin/rm -f

    {} as the argument list marker

    {} is the default argument list marker. You need to use {} this with various command which take more than two arguments at a time. For example mv command need to know the file name. The following will find all .bak files in or below the current directory and move them to ~/.old.files directory:
    $ find . -name "*.bak" -print0 | xargs -0 -I {} mv {} ~/old.files
    You can rename {} to something else. In the following example {} is renamed as file. This is more readable as compare to previous example:
    $ find . -name "*.bak" -print0 | xargs -0 -I file mv file ~/old.files
    Where,

    1. -0 If there are blank spaces or characters (including newlines) many commands will not work. This option take cares of file names with blank space.
    2. -I Replace occurrences of replace-str in the initial-arguments with names read from standard input. Also, unquoted blanks do not terminate input items; instead the separator is the newline character.

    Dealing file names with blank spaces and newline

    The following will work incorrectly if there are any filenames containing newlines or spaces (it will find out all .mp3 file located in current directory and play them using mplayer):
    $ find . -iname "*.mp3" -print | xargs mplayer
    To get rid of this problem use -0 option:
    $ find . -iname "*.mp3" -print0 | xargs -0 -I mp3file mplayer mp3file
    To find out all *.c file located in 100s of subdirectories and move them to another directory called ~/old.src, use:
    $ find /path/to/dir -iname "*.c" -print0 | xargs -0 -I file mv file ~/old.src

    Avoiding errors and resource hungry problems with xargs and find combo

    To copy all media files to another location called /bakup/iscsi, you can use cp as follows:
    $ cp -r -v -p /share/media/mp3/ /backup/iscsi/mp3
    However, cp command may fail if an error occurs such as if the number of files is too large for the cp command to handle. xargs in combination with find can handle such operation nicely. xargs is more resource efficient and will not halt with an error:

    Please note that all of the above commands are tested with GNU/xargs version. BSD and UNIX xargs command may not have options such as -r. Please refer to your local xargs man page for further info:

    man xargs

  • —————–
  • xargs is a just like “awk” ,”find” & “grep” commands processes the standard input on all unix flavoured operating sysems.
    Basically “xargs” is used to remove or do some operation on long list of file names which were produced by “find” & “grep” commands.Usually many UNIX operating system doesn’t accept such a long list of argument.UNIX xargs command divide that list into sub-list with acceptable length and made it work.For example I’d like to find out all *.sh file located in 100s of sub-directories and move them to another directory called ~/back.scripts. How do I use command line args with xargs to achieve the same?as per man page “xargs” is used to execute a command, passing constructed argument list(s). The arguments are typically a long list of filenames (generated by ls or find etc) that are passed to xargs via a pipe.

    Some features:

    • xargs can execute the command supplying some initial arguments directly, and reading the remaining arguments from standard input (or piped input).
    • xargs passes arguments to command in several bundles, this allows command to process more arguments than it could normally handle at once.
    • Arguments in the standard input must be separated by unquoted blank characters, or unescaped blank characters or newline characters.
    • Characters can be quoted by enclosing them in “double-quotes” (non-double-quote and non-newline chars only).
    • Characters can be quoted by enclosing them in ‘apostrophes’ (non-apostrophe and non-newline chars only).
    • Any unquoted character can be escaped by preceding it with a backslash.
    e.g. file1 file2 “file three” ‘file four’ file five
    If command is omitted then the equivalent of /bin/echo is used.

     Exit Status

    This command returns the following exit values:
    Item Description
    0 All invocations of the Command parameter returned exit status 0.
    1-125 A command line meeting the specified requirements could not be assembled, one or more of the invocations of the Command parameter returned a non-zero exit status, or some other error occurred.
    126 Command was found but could not be invoked.
    127 Command could not be found.

    Examples

    Find all the .mp3 files in the music folder and pass to the ls command, -print0 is required if any filenames contain whitespace.:

       find ./music -name “*.mp3” -print0 | xargs -0 ls

    Find all files in the work folder, pass to grep and search for profit:

       find ./work -print | xargs grep “profit”

    {} as the argument list marker

    {} is the default argument list marker. You need to use {} this with various command which take more than two arguments at a time. For example mv command need to know the file name. The following will find all .bak files in or below the current directory and move them to ~/.old.files directory:
    $ find . -name “*.sh” -print0 | xargs -0 -I {} mv {} ~/back.scripts

    You can rename {} to something else. In the following example {} is renamed as file. This is more readable as compare to previous example:

    $ find . -name “*.sh” -print0 | xargs -0 -I file mv file ~/back.scripts

    Where,
    -0 If there are blank spaces or characters (including newlines) many commands will not work. This option take cares of file names with blank space.(用于处理文件名中含有空格或空行等空白字符的情况)
    -I Replace occurrences of replace-str in the initial-arguments with names read from standard input. Also, unquoted blanks do not terminate input items; instead the separator is the newline character.

    10 Popular  “XARGS” Command Examples:

    1) With& Without “xargs” observation:

     in this example of xargs command we will see how output changes with use of xargs command in unix or Linux. Here is the output of find command without xargs first and than with xargs, you can clearly see that multiline output is converted into single line:

    um@server#find . -name “*sh*”
    ./.bash_history
    ./.bash_profile
    ./.bash_profile.cf-before-edit
    ./.cshrc
    ./.cshrc.cf-before-edit
    ./.sh_history
    ./.ssh
    ./.ssh2
    ./scripts/aix_sysinfo.ksh
    ./scripts/chperm_messages.sh
    ./scripts/linux_sysinfo.ksh
    ./scripts/solaris_sysinfo_v1.1.ksh
    ./testlocked.kshum@server# find . -name “*bash*” | xargs
    ./.bash_history ./.bash_profile ./.bash_profile.cf-before-edit ./.cshrc ./.cshrc.cf-before-edit ./.sh_history ./.ssh ./.ssh2 ./scripts/aix_sysinfo.ksh ./scripts/chperm_messages.sh ./scripts/linux_sysinfo.ksh ./scripts/solaris_sysinfo_v1.1.ksh ./testlocked.ksh

    2) Xargs with grep:

    When you use “xargs” in conjusction with find and grep , the grep will look for the specifig word in  each file in the from the stanadard input.

    #find . -name “*.sh” | xargs grep “ksh”

    In the above exanmple first find all .sh  files from current directory or below and than on each .sh file look for word “ksh”.

    3) Covert muti line output into single line

    best example of xargs is  converting output of one command into one line. For example you can run any command and then combine xargs to convert output into single line. here is an example xargs in unix which does that.

    um@server#ls -1 *.sh
    linux_sysinfo.sh
    aix_sysinfo.sh
    audit_script.sh
    chperm_messages.shum@system#ls -1 *.sh | xargs
    linux_sysinfo.sh aix_sysinfo.sh audit_script.sh chperm_messages.sh

    4) To Delete temporary files using xargs & find:

    Another common example of xargs command in unix is removing temporary files from system.

    #find /tmp -name “*.tmp” | xargs rm

    This will remove all .tmp file from /tmp or below directory. xargs in unix is very fast as compared to deleting single file at a time which can also be done by using find command alone

    5)  xargs -0 to handle space in file name

    Above example of xargs command in unix will not work as expected if any of file name contains space or new line on it. To avoid this problem we use find -print0 to produce null separated file name and xargs-0 to handle null separated items. Here is an example of xargs command in unix which can handle file name with spaces and newline:

    #find /tmp -name “*.tmp” -print0 | xargs -0 rm

    6) Counting number of lines/words/characters in each file using xargs & find:

    “find”in conjuction with “xargs” and “wc”  we can count number of lines/words/characters in each file under a perticaular directory.

    um@server#ls -1 *.sh | xargs wc -l
    112 linux_sysinfo.sh
    85  aix_sysinfo.sh
    35  audit_script.sh
    18  chperm_messages.sh
    250 total

    Note: you can use ‘-c’ & ‘-w’ with wc to obtain number of characters and words repectively.

    7) xargs and cut command in Unix:

     Though most of xargs examples in unix will be along with find and grep command but xargs is not just limited to this two it can also be used with any command which generated long list of input for example we can use xargs with cut command in unix. In below example of unix xargs we will xargs example with cut command. for using cut command let’s first create a .csv file with some data e.g.

    um@server# cat fruits.txt
    Orange,Greenorange
    Mango,Redmango
    Banana,PinkbananaNow we will display name of actual fruit from first column using xargs command in one line:um@server:/etc cut -d, -f1 smartphones.csv | sort | xargs
    Orange Mango Banana

    8)To insert file names into the middle of command lines, type:(批量重命名,而且考虑到了文件名中的空白符等特殊字符

    um@server#ls | xargs  -t  -I  {} mv {} {}.oldThis command sequence renames all files in the current directory by adding .old to the end of each name. The -I flag tells the xargs command to insert each line of the ls directory listing where {} (braces) appear. If the current directory contains the files chap1, chap2, and chap3, this constructs the following commands:#mv chap1 chap1.old
    #mv chap2 chap2.old
    #mv chap3 chap3.old

    9) To run a command on files that you select individually, type:

    um@server# ls | xargs  -p  -n  1 ar r lib.aThis command sequence allows you to select files to add to the lib.a library. The -p flag tells the xargs command to display each ar command it constructs and to ask if you want to run it. Type y to run the command. Press the any other key if you do not want to run the command.Something similar to the following displays:ar r lib.a chap1 ?…
    ar r lib.a chap2 ?…
    ar r lib.a chap3 ?…

    10) To construct a command that contains a specific number of arguments and to insert those arguments into the middle of a command line, type:

    um@server# ls | xargs -n6 | xargs -I{} echo {} – some files in the directoryIf the current directory contains files chap1 through chap10, the output constructed will be the following:
    chap1 chap2 chap3 chap4 chap5 chap6 – some files in the directory
    chap7 chap8 chap9 chap10 – some file in the directory

声明: 除非注明,ixyzero.com文章均为原创,转载请以链接形式标明本文地址,谢谢!
https://ixyzero.com/blog/archives/156.html

发表评论

电子邮件地址不会被公开。 必填项已用*标注