首页 > 代码库 > AWK“煎蛋”教程

AWK“煎蛋”教程

2024-07-31 06:46:23   213人阅读

只想达到两个目的：

1）熟悉awk。

2）临近双十一值夜班打发时间。

  • 开始

　　从netstat命令中提取了如下信息作为用例

[root@appstockworker tmp]# netstat >>netstat.txt[root@appstockworker tmp]# cat netstat.txt Active Internet connections (w/o servers)Proto Recv-Q Send-Q Local Address               Foreign Address             State      tcp        0      0 appStockWorker.soa.360b:ssh 10.13.133.61:20799          ESTABLISHED tcp        0      0 appStockWorker.soa.360:http 192.168.195.187:42687       TIME_WAIT   tcp        0      0 appStockWorker.soa.360:http 192.168.195.187:42661       TIME_WAIT   tcp        0      0 appStockWorker.soa.360b:ssh 10.13.133.61:53685          ESTABLISHED tcp        0     52 appStockWorker.soa.360b:ssh 10.13.184.67:60857          ESTABLISHED tcp        0      0 appStockWorker.soa.360:http 192.168.195.187:42686       TIME_WAIT   tcp        0      0 appStockWorker.soa.360b:ssh 192.168.192.104:60765       ESTABLISHED tcp        0      0 appStockWorker.soa.360b:ssh 10.13.184.67:58558          ESTABLISHED tcp        0      0 appStockWorker.soa.36:58902 [UNKNOWN]:61620             ESTABLISHED tcp        0      0 appStockWorker.soa.36:37601 [UNKNOWN]:eforward          ESTABLISHED tcp        0      0 appStockWorker.soa.36:50057 [UNKNOWN]:ncube-lm          TIME_WAIT   tcp        0      0 appStockWorker.soa.36:50465 [UNKNOWN]:36063             ESTABLISHED tcp        0      0 appStockWorker.soa.36:20881 [UNKNOWN]:52311             ESTABLISHED tcp        0      0 appStockWorker.soa.36:60375 [UNKNOWN]:ncube-lm          ESTABLISHED tcp        1      0 appStockWorker.soa.36:40517 purchaseconfig.purchas:http CLOSE_WAIT  tcp        0      0 appStockWorker.soa.36:50077 [UNKNOWN]:ncube-lm          ESTABLISHED tcp        0      0 appStockWorker.soa.36:57027 [UNKNOWN]:ms-sql-s          ESTABLISHED tcp        0      0 appStockWorker.soa.36:39624 [UNKNOWN]:cgn-stat          ESTABLISHED tcp        0      0 appStockWorker.soa.36:50078 [UNKNOWN]:ncube-lm          ESTABLISHED tcp        0      0 appStockWorker.soa.36:39093 [UNKNOWN]:isdnlog           ESTABLISHED tcp        0      0 appStockWorker.soa.36:48835 purchaseconfig.purchas:8016 ESTABLISHED tcp        0      0 appStockWorker.soa.36:36060 [UNKNOWN]:60983             ESTABLISHED tcp        0      0 appStockWorker.soa.36:57039 [UNKNOWN]:ms-sql-s          ESTABLISHED

下面是最简单最常用的awk示例，其输出第1列和第4例，

　　其中单引号中的被大括号括着的就是awk的语句，注意，其只能被单引号包含。

　　其中的$1..$n表示第几例。注：$0表示整个行。

[root@appstockworker tmp]# awk ‘{print $1,$4}‘ netstat.txt Active (w/oProto Localtcp appStockWorker.soa.360b:sshtcp appStockWorker.soa.360:httptcp appStockWorker.soa.360:httptcp appStockWorker.soa.360b:sshtcp appStockWorker.soa.360b:sshtcp appStockWorker.soa.360:httptcp appStockWorker.soa.360b:sshtcp appStockWorker.soa.360b:sshtcp appStockWorker.soa.36:58902tcp appStockWorker.soa.36:37601tcp appStockWorker.soa.36:50057tcp appStockWorker.soa.36:50465tcp appStockWorker.soa.36:20881tcp appStockWorker.soa.36:60375tcp appStockWorker.soa.36:40517tcp appStockWorker.soa.36:50077tcp appStockWorker.soa.36:57027tcp appStockWorker.soa.36:39624tcp appStockWorker.soa.36:50078tcp appStockWorker.soa.36:39093tcp appStockWorker.soa.36:48835tcp appStockWorker.soa.36:36060tcp appStockWorker.soa.36:57039tcp appStockWorker.soa.36:57829tcp appStockWorker.soa.36:44759tcp appStockWorker.soa.36:35197tcp appStockWorker.soa.36:52525tcp appStockWorker.soa.36:37605

 我们再来看看awk的格式化输出，和C语言的printf没什么两样：

tcp appStockWorker.soa.36:33428[root@appstockworker tmp]# awk ‘{printf "%-8s %-8s %-8s %-18s %-30s %-15s\n",$1,$2,$3,$4,$5,$6}‘ netstat.txt Active   Internet connections (w/o               servers)                                      Proto    Recv-Q   Send-Q   Local              Address                        Foreign        tcp      0        0        appStockWorker.soa.360b:ssh 10.13.133.61:20799             ESTABLISHED    tcp      0        0        appStockWorker.soa.360:http 192.168.195.187:42687          TIME_WAIT      tcp      0        0        appStockWorker.soa.360:http 192.168.195.187:42661          TIME_WAIT      tcp      0        0        appStockWorker.soa.360b:ssh 10.13.133.61:53685             ESTABLISHED    tcp      0        52       appStockWorker.soa.360b:ssh 10.13.184.67:60857             ESTABLISHED    tcp      0        0        appStockWorker.soa.360:http 192.168.195.187:42686          TIME_WAIT      tcp      0        0        appStockWorker.soa.360b:ssh 192.168.192.104:60765          ESTABLISHED    tcp      0        0        appStockWorker.soa.360b:ssh 10.13.184.67:58558             ESTABLISHED    tcp      0        0        appStockWorker.soa.36:58902 [UNKNOWN]:61620                ESTABLISHED    tcp      0        0        appStockWorker.soa.36:37601 [UNKNOWN]:eforward             ESTABLISHED    tcp      0        0        appStockWorker.soa.36:50057 [UNKNOWN]:ncube-lm             TIME_WAIT      tcp      0        0        appStockWorker.soa.36:50465 [UNKNOWN]:36063                ESTABLISHED    tcp      0        0        appStockWorker.soa.36:20881 [UNKNOWN]:52311                ESTABLISHED    tcp      0        0        appStockWorker.soa.36:60375 [UNKNOWN]:ncube-lm             ESTABLISHED    tcp      1        0        appStockWorker.soa.36:40517 purchaseconfig.purchas:http    CLOSE_WAIT     tcp      0        0        appStockWorker.soa.36:50077 [UNKNOWN]:ncube-lm             ESTABLISHED    tcp      0        0        appStockWorker.soa.36:57027 [UNKNOWN]:ms-sql-s             ESTABLISHED    tcp      0        0        appStockWorker.soa.36:39624 [UNKNOWN]:cgn-stat             ESTABLISHED    tcp      0        0        appStockWorker.soa.36:50078 [UNKNOWN]:ncube-lm             ESTABLISHED    tcp      0        0        appStockWorker.soa.36:39093 [UNKNOWN]:isdnlog              ESTABLISHED
  • 过滤记录

我们再来看看如何过滤记录（下面过滤条件为：第三列的值为0 && 第6列的值为LISTEN）

[root@appstockworker tmp]# awk ‘$3==0 && $6=="TIME_WAIT"‘ netstat.txt tcp        0      0 appStockWorker.soa.360:http 192.168.195.187:42687       TIME_WAIT   tcp        0      0 appStockWorker.soa.360:http 192.168.195.187:42661       TIME_WAIT   tcp        0      0 appStockWorker.soa.360:http 192.168.195.187:42686       TIME_WAIT   tcp        0      0 appStockWorker.soa.36:50057 [UNKNOWN]:ncube-lm          TIME_WAIT   tcp        0      0 localhost.localdomain:8010  localhost.localdomain:43856 TIME_WAIT   tcp        0      0 appStockWorker.soa.36:50056 [UNKNOWN]:ncube-lm          TIME_WAIT   tcp        0      0 appStockWorker.soa.36:57017 [UNKNOWN]:ms-sql-s          TIME_WAIT   tcp        0      0 appStockWorker.soa.36:50046 [UNKNOWN]:ncube-lm          TIME_WAIT   tcp        0      0 appStockWorker.soa.36:50068 [UNKNOWN]:ncube-lm          TIME_WAIT   tcp        0      0 appStockWorker.soa.36:57005 [UNKNOWN]:ms-sql-s          TIME_WAIT   tcp        0      0 appStockWorker.soa.36:50047 [UNKNOWN]:ncube-lm          TIME_WAIT   tcp        0      0 appStockWorker.soa.36:50069 [UNKNOWN]:ncube-lm          TIME_WAIT   tcp        0      0 appStockWorker.soa.36:56996 [UNKNOWN]:ms-sql-s          TIME_WAIT   tcp        0      0 localhost.localdomain:8010  localhost.localdomain:43852 TIME_WAIT   tcp        0      0 localhost.localdomain:8010  localhost.localdomain:43855 TIME_WAIT

其中的“==”为比较运算符。其他比较运算符：!=, >, <, >=, <=

我们来看看各种过滤记录的方式：

[root@appstockworker tmp]# awk ‘$3>0 {print $0}‘ netstat.txt Active Internet connections (w/o servers)Proto Recv-Q Send-Q Local Address               Foreign Address             State      tcp        0     52 appStockWorker.soa.360b:ssh 10.13.184.67:60857          ESTABLISHED tcp        0      1 appStockWorker.soa.36:59174 [UNKNOWN]:20880             SYN_SENT

如果我们需要表头的话，我们可以引入内建变量NR：

[root@appstockworker tmp]# awk ‘$3==0 && $6=="TIME_WAIT" || NR==2 ‘ netstat.txt Proto Recv-Q Send-Q Local Address               Foreign Address             State      tcp        0      0 appStockWorker.soa.360:http 192.168.195.187:42687       TIME_WAIT   tcp        0      0 appStockWorker.soa.360:http 192.168.195.187:42661       TIME_WAIT   tcp        0      0 appStockWorker.soa.360:http 192.168.195.187:42686       TIME_WAIT   tcp        0      0 appStockWorker.soa.36:50057 [UNKNOWN]:ncube-lm          TIME_WAIT   tcp        0      0 localhost.localdomain:8010  localhost.localdomain:43856 TIME_WAIT   tcp        0      0 appStockWorker.soa.36:50056 [UNKNOWN]:ncube-lm          TIME_WAIT   tcp        0      0 appStockWorker.soa.36:57017 [UNKNOWN]:ms-sql-s          TIME_WAIT   tcp        0      0 appStockWorker.soa.36:50046 [UNKNOWN]:ncube-lm          TIME_WAIT   tcp        0      0 appStockWorker.soa.36:50068 [UNKNOWN]:ncube-lm          TIME_WAIT   tcp        0      0 appStockWorker.soa.36:57005 [UNKNOWN]:ms-sql-s          TIME_WAIT   tcp        0      0 appStockWorker.soa.36:50047 [UNKNOWN]:ncube-lm          TIME_WAIT   tcp        0      0 appStockWorker.soa.36:50069 [UNKNOWN]:ncube-lm          TIME_WAIT   tcp        0      0 appStockWorker.soa.36:56996 [UNKNOWN]:ms-sql-s          TIME_WAIT   tcp        0      0 localhost.localdomain:8010  localhost.localdomain:43852 TIME_WAIT   tcp        0      0 localhost.localdomain:8010  localhost.localdomain:43855 TIME_WAIT

再加上格式化输出：

[root@appstockworker tmp]# awk ‘$3==0 && $6=="TIME_WAIT" || NR==2 {printf "%-20s %-30s %s\n",$4,$5,$6}‘ netstat.txt Local                Address                        ForeignappStockWorker.soa.360:http 192.168.195.187:42687          TIME_WAITappStockWorker.soa.360:http 192.168.195.187:42661          TIME_WAITappStockWorker.soa.360:http 192.168.195.187:42686          TIME_WAITappStockWorker.soa.36:50057 [UNKNOWN]:ncube-lm             TIME_WAITlocalhost.localdomain:8010 localhost.localdomain:43856    TIME_WAITappStockWorker.soa.36:50056 [UNKNOWN]:ncube-lm             TIME_WAITappStockWorker.soa.36:57017 [UNKNOWN]:ms-sql-s             TIME_WAITappStockWorker.soa.36:50046 [UNKNOWN]:ncube-lm             TIME_WAITappStockWorker.soa.36:50068 [UNKNOWN]:ncube-lm             TIME_WAITappStockWorker.soa.36:57005 [UNKNOWN]:ms-sql-s             TIME_WAITappStockWorker.soa.36:50047 [UNKNOWN]:ncube-lm             TIME_WAITappStockWorker.soa.36:50069 [UNKNOWN]:ncube-lm             TIME_WAITappStockWorker.soa.36:56996 [UNKNOWN]:ms-sql-s             TIME_WAITlocalhost.localdomain:8010 localhost.localdomain:43852    TIME_WAITlocalhost.localdomain:8010 localhost.localdomain:43855    TIME_WAIT
  • 内建变量
$0当前记录（这个变量中存放着整个行的内容）
$1~$n当前记录的第n个字段，字段间由FS分隔
FS输入字段分隔符 默认是空格或Tab
NF当前记录中的字段个数，就是有多少列
NR已经读出的记录数，就是行号，从1开始，如果有多个文件话，这个值也是不断累加中。
FNR当前记录数，与NR不同的是，这个值会是各个文件自己的行号
RS输入的记录分隔符， 默认为换行符
OFS输出字段分隔符， 默认也是空格
ORS输出的记录分隔符，默认为换行符
FILENAME当前输入文件的名字

怎么使用呢，比如：我们如果要输出行号：

[root@appstockworker tmp]# awk ‘$3==0 && $6=="TIME_WAIT" || NR==2 {printf "%02s %s %-20s %-30s %s\n",NR,FNR,$4,$5,$6}‘ netstat.txt 02 2 Local                Address                        Foreign04 4 appStockWorker.soa.360:http 192.168.195.187:42687          TIME_WAIT05 5 appStockWorker.soa.360:http 192.168.195.187:42661          TIME_WAIT08 8 appStockWorker.soa.360:http 192.168.195.187:42686          TIME_WAIT13 13 appStockWorker.soa.36:50057 [UNKNOWN]:ncube-lm             TIME_WAIT32 32 localhost.localdomain:8010 localhost.localdomain:43856    TIME_WAIT42 42 appStockWorker.soa.36:50056 [UNKNOWN]:ncube-lm             TIME_WAIT60 60 appStockWorker.soa.36:57017 [UNKNOWN]:ms-sql-s             TIME_WAIT63 63 appStockWorker.soa.36:50046 [UNKNOWN]:ncube-lm             TIME_WAIT66 66 appStockWorker.soa.36:50068 [UNKNOWN]:ncube-lm             TIME_WAIT70 70 appStockWorker.soa.36:57005 [UNKNOWN]:ms-sql-s             TIME_WAIT103 103 appStockWorker.soa.36:50047 [UNKNOWN]:ncube-lm             TIME_WAIT108 108 appStockWorker.soa.36:50069 [UNKNOWN]:ncube-lm             TIME_WAIT112 112 appStockWorker.soa.36:56996 [UNKNOWN]:ms-sql-s             TIME_WAIT117 117 localhost.localdomain:8010 localhost.localdomain:43852    TIME_WAIT127 127 localhost.localdomain:8010 localhost.localdomain:43855    TIME_WAIT
  •  指定分隔符
[root@appstockworker tmp]# awk ‘BEGIN{FS=":"} {print $1,$3,$6}‘ /etc/passwdroot 0 /rootbin 1 /bindaemon 2 /sbinadm 3 /var/admlp 4 /var/spool/lpdsync 5 /sbinshutdown 6 /sbinhalt 7 /sbinmail 8 /var/spool/mailuucp 10 /var/spool/uucpoperator 11 /rootgames 12 /usr/gamesgopher 13 /var/gopherftp 14 /var/ftpnobody 99 /vcsa 69 /devsaslauth 499 /var/empty/saslauthpostfix 89 /var/spool/postfixsshd 74 /var/empty/sshdntp 38 /etc/ntpadmin 500 /home/adminnagios 501 /home/nagios

上面的命令也等价于：（-F的意思就是指定分隔符）

[root@appstockworker tmp]# awk -F: ‘{print $1,$3,$6}‘ /etc/passwd

注：如果你要指定多个分隔符，你可以这样来：

awk -F ‘[;:]‘

再来看一个以\t作为分隔符输出的例子（下面使用了/etc/passwd文件，这个文件是以:分隔的）：

[root@appstockworker tmp]# awk -F: ‘{print $1,$3,$6}‘ OFS="\t" /etc/passwdroot    0       /rootbin     1       /bindaemon  2       /sbinadm     3       /var/admlp      4       /var/spool/lpdsync    5       /sbinshutdown        6       /sbinhalt    7       /sbinmail    8       /var/spool/mailuucp    10      /var/spool/uucpoperator        11      /rootgames   12      /usr/gamesgopher  13      /var/gopherftp     14      /var/ftpnobody  99      /vcsa    69      /devsaslauth        499     /var/empty/saslauthpostfix 89      /var/spool/postfixsshd    74      /var/empty/sshdntp     38      /etc/ntpadmin   500     /home/adminnagios  501     /home/nagios
  • 字符串匹配

我们再来看几个字符串匹配的示例：

[root@appstockworker tmp]# awk ‘$6 ~ /TIME/ || NR==1 {print NR,$4,$5,$6}‘  OFS="\t" netstat.txt 1       (w/o    servers)4       appStockWorker.soa.360:http     192.168.195.187:42687   TIME_WAIT5       appStockWorker.soa.360:http     192.168.195.187:42661   TIME_WAIT8       appStockWorker.soa.360:http     192.168.195.187:42686   TIME_WAIT13      appStockWorker.soa.36:50057     [UNKNOWN]:ncube-lm      TIME_WAIT32      localhost.localdomain:8010      localhost.localdomain:43856     TIME_WAIT42      appStockWorker.soa.36:50056     [UNKNOWN]:ncube-lm      TIME_WAIT60      appStockWorker.soa.36:57017     [UNKNOWN]:ms-sql-s      TIME_WAIT63      appStockWorker.soa.36:50046     [UNKNOWN]:ncube-lm      TIME_WAIT66      appStockWorker.soa.36:50068     [UNKNOWN]:ncube-lm      TIME_WAIT70      appStockWorker.soa.36:57005     [UNKNOWN]:ms-sql-s      TIME_WAIT103     appStockWorker.soa.36:50047     [UNKNOWN]:ncube-lm      TIME_WAIT108     appStockWorker.soa.36:50069     [UNKNOWN]:ncube-lm      TIME_WAIT112     appStockWorker.soa.36:56996     [UNKNOWN]:ms-sql-s      TIME_WAIT117     localhost.localdomain:8010      localhost.localdomain:43852     TIME_WAIT127     localhost.localdomain:8010      localhost.localdomain:43855     TIME_WAIT

上面的第一个示例匹配FIN状态,其实 ~ 表示模式开始。/ /中是模式。这就是一个正则表达式的匹配。

其实awk可以像grep一样的去匹配第一行，就像这样：

[root@appstockworker tmp]# awk ‘/CLOSE_WAIT/‘ netstat.txt tcp        1      0 appStockWorker.soa.36:40517 purchaseconfig.purchas:http CLOSE_WAIT  tcp        1      0 appStockWorker.soa.36:57829 purchaseconfig.purchas:http CLOSE_WAIT  tcp        1      0 appStockWorker.soa.36:44759 purchaseconfig.purchas:http CLOSE_WAIT  tcp        1      0 appStockWorker.soa.36:42397 purchaseconfig.purchas:http CLOSE_WAIT  tcp        1      0 appStockWorker.soa.36:46115 purchaseconfig.purchas:http CLOSE_WAIT  tcp        1      0 appStockWorker.soa.36:36460 purchaseconfig.purchas:8016 CLOSE_WAIT  tcp        1      0 appStockWorker.soa.36:52790 purchaseconfig.purchas:8016 CLOSE_WAIT  tcp        1      0 appStockWorker.soa.36:34745 purchaseconfig.purchas:http CLOSE_WAIT  tcp        1      0 appStockWorker.soa.36:59139 purchaseconfig.purchas:http CLOSE_WAIT  tcp        1      0 appStockWorker.soa.36:37024 purchaseconfig.purchas:http CLOSE_WAIT  tcp        1      0 appStockWorker.soa.36:34171 purchaseconfig.purchas:http CLOSE_WAIT  tcp        1      0 appStockWorker.soa.36:47671 purchaseconfig.purchas:http CLOSE_WAIT

我们可以使用 “/CLOSE|TIME/” 来匹配 CLOSE 或者 TIME :

[root@appstockworker tmp]# awk ‘$6 ~ /CLOSE|TIME/ || NR==1 {print NR,$4,$5,$6}‘  OFS="\t" netstat.txt 1       (w/o    servers)4       appStockWorker.soa.360:http     192.168.195.187:42687   TIME_WAIT5       appStockWorker.soa.360:http     192.168.195.187:42661   TIME_WAIT8       appStockWorker.soa.360:http     192.168.195.187:42686   TIME_WAIT13      appStockWorker.soa.36:50057     [UNKNOWN]:ncube-lm      TIME_WAIT17      appStockWorker.soa.36:40517     purchaseconfig.purchas:http     CLOSE_WAIT26      appStockWorker.soa.36:57829     purchaseconfig.purchas:http     CLOSE_WAIT27      appStockWorker.soa.36:44759     purchaseconfig.purchas:http     CLOSE_WAIT32      localhost.localdomain:8010      localhost.localdomain:43856     TIME_WAIT36      appStockWorker.soa.36:42397     purchaseconfig.purchas:http     CLOSE_WAIT39      appStockWorker.soa.36:46115     purchaseconfig.purchas:http     CLOSE_WAIT42      appStockWorker.soa.36:50056     [UNKNOWN]:ncube-lm      TIME_WAIT44      appStockWorker.soa.36:36460     purchaseconfig.purchas:8016     CLOSE_WAIT47      appStockWorker.soa.36:52790     purchaseconfig.purchas:8016     CLOSE_WAIT48      appStockWorker.soa.36:34745     purchaseconfig.purchas:http     CLOSE_WAIT60      appStockWorker.soa.36:57017     [UNKNOWN]:ms-sql-s      TIME_WAIT63      appStockWorker.soa.36:50046     [UNKNOWN]:ncube-lm      TIME_WAIT66      appStockWorker.soa.36:50068     [UNKNOWN]:ncube-lm      TIME_WAIT70      appStockWorker.soa.36:57005     [UNKNOWN]:ms-sql-s      TIME_WAIT75      appStockWorker.soa.36:59139     purchaseconfig.purchas:http     CLOSE_WAIT88      appStockWorker.soa.36:37024     purchaseconfig.purchas:http     CLOSE_WAIT100     appStockWorker.soa.36:34171     purchaseconfig.purchas:http     CLOSE_WAIT103     appStockWorker.soa.36:50047     [UNKNOWN]:ncube-lm      TIME_WAIT108     appStockWorker.soa.36:50069     [UNKNOWN]:ncube-lm      TIME_WAIT112     appStockWorker.soa.36:56996     [UNKNOWN]:ms-sql-s      TIME_WAIT117     localhost.localdomain:8010      localhost.localdomain:43852     TIME_WAIT123     appStockWorker.soa.36:47671     purchaseconfig.purchas:http     CLOSE_WAIT127     localhost.localdomain:8010      localhost.localdomain:43855     TIME_WAIT

再来看看模式取反的例子：

[root@appstockworker tmp]# awk ‘$6 !~ /WAIT/ || NR==1 {print NR,$4,$5,$6}‘ OFS="\t" netstat.txt 1       (w/o    servers)2       Local   Address Foreign3       appStockWorker.soa.360b:ssh     10.13.133.61:20799      ESTABLISHED6       appStockWorker.soa.360b:ssh     10.13.133.61:53685      ESTABLISHED7       appStockWorker.soa.360b:ssh     10.13.184.67:60857      ESTABLISHED9       appStockWorker.soa.360b:ssh     192.168.192.104:60765   ESTABLISHED10      appStockWorker.soa.360b:ssh     10.13.184.67:58558      ESTABLISHED11      appStockWorker.soa.36:58902     [UNKNOWN]:61620 ESTABLISHED12      appStockWorker.soa.36:37601     [UNKNOWN]:eforward      ESTABLISHED14      appStockWorker.soa.36:50465     [UNKNOWN]:36063 ESTABLISHED15      appStockWorker.soa.36:20881     [UNKNOWN]:52311 ESTABLISHED16      appStockWorker.soa.36:60375     [UNKNOWN]:ncube-lm      ESTABLISHED18      appStockWorker.soa.36:50077     [UNKNOWN]:ncube-lm      ESTABLISHED19      appStockWorker.soa.36:57027     [UNKNOWN]:ms-sql-s      ESTABLISHED20      appStockWorker.soa.36:39624     [UNKNOWN]:cgn-stat      ESTABLISHED21      appStockWorker.soa.36:50078     [UNKNOWN]:ncube-lm      ESTABLISHED22      appStockWorker.soa.36:39093     [UNKNOWN]:isdnlog       ESTABLISHED23      appStockWorker.soa.36:48835     purchaseconfig.purchas:8016     ESTABLISHED

或是

awk ‘!/WAIT/‘ netstat.txt
  •  折分文件

awk拆分文件很简单，使用重定向就好了。下面这个例子，是按第6例分隔文件，相当的简单（其中的NR!=1表示不处理表头）。

[root@appstockworker tmp]# awk ‘NR!=1{print >$6}‘ netstat.txt[root@appstockworker tmp]# lsaaa.sh                              hudson-remoting2477836264792765148  hudson-remoting5656432457920204714  hudson-remoting9133536725345626708CLOSE_WAIT                          hudson-remoting267624195045038773   hudson-remoting5738546470805781773  hudson-remoting9195511106968949486ESTABLISHED                         hudson-remoting2782799830874667135  hudson-remoting5862972180048370977  jnaForeign                             hudson-remoting3026235916867749211  hudson-remoting6048063392649864890  netstat.txthsperfdata_admin                    hudson-remoting3433616259928789742  hudson-remoting6507341373611789601  ssh-FhWcBH2899hsperfdata_root                     hudson-remoting3493246008991492784  hudson-remoting6509990390611516239  ssh-WwilYy6121hudson-remoting1020802418708175254  hudson-remoting3691137831114289115  hudson-remoting796125568617163193   SYN_SENThudson-remoting1288644704899054285  hudson-remoting4268187764899187207  hudson-remoting7989065960669590914  TIME_WAIThudson-remoting135955832302030064   hudson-remoting4541529833790610464  hudson-remoting8053604203671609757  yum.loghudson-remoting1594902447089122032  hudson-remoting4709407012955215290  hudson-remoting8134591377726690969  zbuilder_deploy.shhudson-remoting1657148174411050811  hudson-remoting4718570550800607052  hudson-remoting8297456844425463707hudson-remoting2051518431688100965  hudson-remoting5098346600070636216  hudson-remoting8859803414006288608[root@appstockworker tmp]# cat CLOSE_WAITtcp        1      0 appStockWorker.soa.36:40517 purchaseconfig.purchas:http CLOSE_WAIT  tcp        1      0 appStockWorker.soa.36:57829 purchaseconfig.purchas:http CLOSE_WAIT  tcp        1      0 appStockWorker.soa.36:44759 purchaseconfig.purchas:http CLOSE_WAIT  tcp        1      0 appStockWorker.soa.36:42397 purchaseconfig.purchas:http CLOSE_WAIT  tcp        1      0 appStockWorker.soa.36:46115 purchaseconfig.purchas:http CLOSE_WAIT  tcp        1      0 appStockWorker.soa.36:36460 purchaseconfig.purchas:8016 CLOSE_WAIT  tcp        1      0 appStockWorker.soa.36:52790 purchaseconfig.purchas:8016 CLOSE_WAIT  tcp        1      0 appStockWorker.soa.36:34745 purchaseconfig.purchas:http CLOSE_WAIT  tcp        1      0 appStockWorker.soa.36:59139 purchaseconfig.purchas:http CLOSE_WAIT  tcp        1      0 appStockWorker.soa.36:37024 purchaseconfig.purchas:http CLOSE_WAIT  tcp        1      0 appStockWorker.soa.36:34171 purchaseconfig.purchas:http CLOSE_WAIT  tcp        1      0 appStockWorker.soa.36:47671 purchaseconfig.purchas:http CLOSE_WAIT  [root@appstockworker tmp]# cat TIME_WAITtcp        0      0 appStockWorker.soa.360:http 192.168.195.187:42687       TIME_WAIT   tcp        0      0 appStockWorker.soa.360:http 192.168.195.187:42661       TIME_WAIT   tcp        0      0 appStockWorker.soa.360:http 192.168.195.187:42686       TIME_WAIT   tcp        0      0 appStockWorker.soa.36:50057 [UNKNOWN]:ncube-lm          TIME_WAIT   tcp        0      0 localhost.localdomain:8010  localhost.localdomain:43856 TIME_WAIT   tcp        0      0 appStockWorker.soa.36:50056 [UNKNOWN]:ncube-lm          TIME_WAIT   tcp        0      0 appStockWorker.soa.36:57017 [UNKNOWN]:ms-sql-s          TIME_WAIT   tcp        0      0 appStockWorker.soa.36:50046 [UNKNOWN]:ncube-lm          TIME_WAIT   tcp        0      0 appStockWorker.soa.36:50068 [UNKNOWN]:ncube-lm          TIME_WAIT   tcp        0      0 appStockWorker.soa.36:57005 [UNKNOWN]:ms-sql-s          TIME_WAIT   tcp        0      0 appStockWorker.soa.36:50047 [UNKNOWN]:ncube-lm          TIME_WAIT   tcp        0      0 appStockWorker.soa.36:50069 [UNKNOWN]:ncube-lm          TIME_WAIT   tcp        0      0 appStockWorker.soa.36:56996 [UNKNOWN]:ms-sql-s          TIME_WAIT   tcp        0      0 localhost.localdomain:8010  localhost.localdomain:43852 TIME_WAIT   tcp        0      0 localhost.localdomain:8010  localhost.localdomain:43855 TIME_WAIT

你也可以把指定的列输出到文件：

awk ‘NR!=1{print $4,$5 >$6}‘ netstat.txt

再复杂一点：（注意其中的if-else-if语句，可见awk其实是个脚本解释器）

[root@appstockworker tmp]# awk ‘NR!=1{if($6 ~/TIME|ESTABLISHED/) print >"1.txt";else if($6 ~/CLOSE/) print >"2.txt";else print > "3.txt"}‘ netstat.txt

[root@appstockworker tmp]# cat 2.txt
tcp 1 0 appStockWorker.soa.36:40517 purchaseconfig.purchas:http CLOSE_WAIT
tcp 1 0 appStockWorker.soa.36:57829 purchaseconfig.purchas:http CLOSE_WAIT
tcp 1 0 appStockWorker.soa.36:44759 purchaseconfig.purchas:http CLOSE_WAIT
tcp 1 0 appStockWorker.soa.36:42397 purchaseconfig.purchas:http CLOSE_WAIT
tcp 1 0 appStockWorker.soa.36:46115 purchaseconfig.purchas:http CLOSE_WAIT
tcp 1 0 appStockWorker.soa.36:36460 purchaseconfig.purchas:8016 CLOSE_WAIT
tcp 1 0 appStockWorker.soa.36:52790 purchaseconfig.purchas:8016 CLOSE_WAIT
tcp 1 0 appStockWorker.soa.36:34745 purchaseconfig.purchas:http CLOSE_WAIT
tcp 1 0 appStockWorker.soa.36:59139 purchaseconfig.purchas:http CLOSE_WAIT
tcp 1 0 appStockWorker.soa.36:37024 purchaseconfig.purchas:http CLOSE_WAIT
tcp 1 0 appStockWorker.soa.36:34171 purchaseconfig.purchas:http CLOSE_WAIT
tcp 1 0 appStockWorker.soa.36:47671 purchaseconfig.purchas:http CLOSE_WAIT
[root@appstockworker tmp]# cat 3.txt
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 1 appStockWorker.soa.36:59174 [UNKNOWN]:20880 SYN_SENT

  • 统计

下面的命令计算所有的C文件，CPP文件和H文件的文件大小总和。

[root@appstockworker tmp]# ls -l *.txt *.log *.sh| awk ‘{sum+=$5} END {print sum}‘42044

我们再来看一个统计各个connection状态的用法：注意其中的数组的用法

[root@appstockworker tmp]# awk ‘NR!=1{a[$6]++;} END {for (i in a) print i "," a[i];}‘ netstat.txt TIME_WAIT,15CLOSE_WAIT,12SYN_SENT,1ESTABLISHED,105Foreign,1

再来看看统计每个用户的进程的占了多少内存（注：sum的RSS那一列）

[root@appstockworker tmp]# ps aux | awk ‘NR!=1{a[$1]+=$6;} END { for(i in a) print i ", " a[i]"KB";}‘nagios, 480KBadmin, 4247804KBpostfix, 4692KBroot, 157908KB
  • awk脚本

在上面我们可以看到一个END关键字。END的意思是“处理完所有的行的标识”，即然说到了END就有必要介绍一下BEGIN，这两个关键字意味着执行前和执行后的意思，语法如下：

  • BEGIN{ 这里面放的是执行前的语句 }
  • END {这里面放的是处理完所有的行后要执行的语句 }
  • {这里面放的是处理每一行时要执行的语句}

为了说清楚这个事，我们来看看下面的示例：

假设有这么一个文件（学生成绩表）：

[root@appstockworker tmp]# cat score.txt Marry   2143 78 84 77Jack    2321 66 78 45Tom     2122 48 77 71Mike    2537 87 97 95Bob     2415 40 57 62[root@appstockworker tmp]# vi cal.awk#!/bin/awk -f#运行前BEGIN{        math = 0        english = 0        computer = 0        printf "NAME    NO.   MATH  ENGLISH  COMPUTER   TOTAL\n"        printf "---------------------------------------------\n"}#运行中{        math+=$3        english+=$4        computer+=$5        printf "%-6s %-6s %4d %8d %8d %8d\n", $1, $2, $3,$4,$5, $3+$4+$5}#运行后END{        printf "---------------------------------------------\n"        printf "  TOTAL:%10d %8d %8d \n", math, english, computer        printf "AVERAGE:%10.2f %8.2f %8.2f\n", math/NR, english/NR, computer/NR}

我们来看一下执行结果：（也可以这样运行 ./cal.awk score.txt）

[root@appstockworker tmp]# awk -f cal.awk score.txt NAME    NO.   MATH  ENGLISH  COMPUTER   TOTAL---------------------------------------------Marry  2143     78       84       77      239Jack   2321     66       78       45      189Tom    2122     48       77       71      196Mike   2537     87       97       95      279Bob    2415     40       57       62      159---------------------------------------------  TOTAL:       319      393      350 AVERAGE:     63.80    78.60    70.00
  • 环境变量

即然说到了脚本，我们来看看怎么和环境变量交互：（使用-v参数和ENVIRON，使用ENVIRON的环境变量需要export）

[root@appstockworker tmp]# x=5[root@appstockworker tmp]# y=10[root@appstockworker tmp]# export y[root@appstockworker tmp]# echo $x $y5 10[root@appstockworker tmp]# awk -v val=$x ‘{print $1,$2,$3,$4+val,$5+ENVIRON["y"]}‘ OFS="\t" score.txt Marry   2143    78      89      87Jack    2321    66      83      55Tom     2122    48      82      81Mike    2537    87      102     105Bob     2415    40      62      72
  •  几个例子
#从file文件中找出长度大于80的行awk ‘length>80‘ file#按连接数查看客户端IP[root@appstockworker tmp]# netstat -ntu | awk NR!=1‘{print $5}‘ | cut -d: -f1 | sort | uniq -c | sort -nr    125       5 192.168.195.187      2 10.13.184.67      2 10.13.133.61      1 Address      1 192.168.192.104#打印99乘法表[root@appstockworker tmp]# seq 9 | sed ‘H;g‘ | awk -v RS=‘‘ ‘{for(i=1;i<=NF;i++)printf("%dx%d=%d%s", i, NR, i*NR, i==NR?"\n":"\t")}‘1x1=11x2=2   2x2=41x3=3   2x3=6   3x3=91x4=4   2x4=8   3x4=12  4x4=161x5=5   2x5=10  3x5=15  4x5=20  5x5=251x6=6   2x6=12  3x6=18  4x6=24  5x6=30  6x6=361x7=7   2x7=14  3x7=21  4x7=28  5x7=35  6x7=42  7x7=491x8=8   2x8=16  3x8=24  4x8=32  5x8=40  6x8=48  7x8=56  8x8=641x9=9   2x9=18  3x9=27  4x9=36  5x9=45  6x9=54  7x9=63  8x9=72  9x9=81

 

AWK“煎蛋”教程


联系
我们