AWK 入門

AWK 入門 Advisor: Quincy Wu Speaker: Kuan-Ta Lu Date: July 8, 2010

What Is AWK? • 是一種可以處理資料、產生格式化報表的語言，功能相當強大。 • awk的工作方式是讀取資料檔，將每一列資料視為一筆記錄（record），每筆記錄以欄位分隔符號分成若干欄位，然後，輸出各個欄位的值。 • 需要具備正規表示式的基本知識。

awk 的用法 (1) • awk '條件類型1{動作1} 條件類型2{動作2} ...' filename • $ awk '{ print }' /etc/passwd • $ awk -F":" '{ print $1 }' /etc/passwd • $ ls -l /etc | awk '{print $1 "\t" $9}'

awk 的用法 (2) • $ awk '{print $2,$3}' emp_names • $ awk '/AL/ {print $3,$2}' emp_names • $ awk '/AL/ {print $3,$2 ; print $4,$5}' emp_names • $ awk '/AL/ {print $3,$2} {print $4,$5}' emp_names • $ awk '/AL/ {print $3,$2 ; print $4", "$5"\n"}' emp_names

awk 的用法 (3) • $ awk '/AL|IN/' emp_names • $ awk '$5 ˜ /AR/' emp_names • $ awk '$5 !˜ /AR/' emp_names

awk 的變數 (1) • 變數 • NF 每一行 ($0) 擁有的欄位總數 • NR 目前 awk 所處理的是『第幾行』資料 • FS 目前的分隔字元，預設是空白鍵 • RS 記錄的分隔字元，預設是行 • OFS 輸出分隔字元 • ORS 輸出記錄分隔字元，預設是行 • $ last | awk '{print $1 "\t lines: " NR "\t columes: " NF}'

awk 的變數 (2) • $ awk '{FS=":"}{print $2}' mou_names • $ awk -F":" '{OFS="-"}{print $1,$2,$3,$4,$5}' mou_names • $ awk -F":" '{print NR,$1,$2,$3}' mou_names • $ awk -F":" '/4601[2-5]/' mou_names

awk 的運算 (1) • 邏輯運算 • > 大於 • < 小於 • >= 大於或等於 • <= 小於或等於 • == 等於 • != 不等於 • 數學運算 • + adds numbers together • - subtracts • * multiplies • / divides

awk 的運算 (2) • ^ performs exponential mathematics • % gives the modulo • ++ adds one to the value of a variable • += assigns the result of an addition operation to a variable • -- subtracts one from a variable • -= assigns the result of a subtraction operation to a variable • *= assigns the result of multiplication • /= assigns the result of division • %= assigns the result of a modulo operation • cat /etc/passwd | awk '{FS=":"} $3 < 10 {print $1 "\t " $3}'

awk 的運算 (3) • $ cat pay | awk 'NR==1{printf "%10s %10s %10s %10s %10s\n",$1,$2,$3,$4,"Total"} NR>=2{total=$2+$3+$4 printf "%10s %10d %10d %10d %10.2f\n",$1,$2,$3,$4,total}‘ • $ awk '{print $1,"QTY: "$2,"PRICE: "$3,"TOTAL: "$2*$3}' inventory

awk 的運算 (4) • $ awk '{x=x+$2} {print x}' inventory • $ awk '{x=x+($2*$3)}{print $1,"QTY: "$2,"PRICE: "$3,"TOTAL: "$2*$3,"BAL: "x}' inventory • $ awk '{x=x+($2*$3)} {print $1,"QTY: "$2,"PRICE: "$3,"TOTAL: "$2*$3} END {print "Total Value of Inventory: " x}' inventory

awk 的運算 (5) • $ sort emp_names | awk '{print $3,$2}' • $ awk '{print NR, $1 > "/tmp/filez" }' emp_names • $ awk '{ print $2 | "sort" }' emp_names

Example (1) $ cat N # just a test 33 22 999 223 44 55 99 $ ./g-than N # just a test 999 223 99 $ cat g-than #!/bin/awk -f BEGIN{ lowerbound = 55 } /^#/ { print $0 } ! /^#/ { for(i=1; i<=NF; i++) if( $i > lowerbound ) print $i }

Example (2) $ cat > IN 233 112 33 # oh!, just a test # again 333 hi kk 22 $ ./sum-fields IN =378 # oh!, just a test # again hikk=355 $ cat sum-fields #!/bin/awk -f /^#/ { print $0 } ! /^#/ { name = "" sum = 0 for(i=1; i<=NF; i++) if( $i ~ /[0-9][-9]*/ ) sum = sum + $i else name = name $i print name "=" sum }

Example (3) $ cat > IN-num klim 233 112 33 # oh!, just a test # again milk 333 22 klim 999 22 oak 222 redcow 1023 oak 11 milk 333 22 $ ./sum-vars IN-num # oh!, just a test # again redcow = 1023 klim = 1399 oak = 233 milk = 710 $ cat > sum-vars #!/bin/awk -f /^#/ { print $0 } ! /^#/ { name = $1 sum = 0 for(i=2; i<=NF; i++) sum += $i var[ name ] += sum } END{ for (v in var) printf "%s = %d\n", v, var[v] }

Exercises • /etc/passwd 以冒號 “:“ 作為欄位的分隔，如何查閱第三欄小於等於 50 以下的數據，並且僅列出帳號與第三欄以tab隔開？ • 利用awk計算出檔案sshreport.pl有幾行空白行？ • 寫一個awk script, 功能如下: $./ex.awk ex-input oak 355 klim 9019 ryan 73 redcow 40000 $ cat ex-input # klim 23 33 22 11 8888 42 # oak 1000 200 2200 -22 -3000 -20 -3 # redcow 20000 20000 # ryan 73 94 -94

Reference • http://www.oracle.com/technology/pub/articles/dulaney_awk.html • http://www.cyut.edu.tw/~dywang/linuxProgram/node20.html • http://www.vectorsite.net/tsawk_2.html • http://www.study-area.org/cyril/scripts/scripts/node61.html • http://www.grymoire.com/Unix/Awk.html • http://nixchun.pixnet.net/blog/post/21760238 • http://erdos.csie.ncnu.edu.tw/~klim/

AWK 入門

AWK 入門

Presentation Transcript