what info can you get from log files how do you get it october 11 2010
Skip this Video
Download Presentation

Loading in 2 Seconds...

play fullscreen
1 / 35

title: - PowerPoint PPT Presentation

  • Uploaded on

What info can you get from log files? How do you get it? October 11, 2010. title:. Examples. How many dynamic address clients do we have in a day? How many address leases have been used? Are any of our clients taking more than their fair share of addresses (i.e. one)?

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' title:' - dustin

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
  • How many dynamic address clients do we have in a day?
  • How many address leases have been used?
  • Are any of our clients taking more than their fair share of addresses (i.e. one)?
  • How many iPods are on cruznet?
  • How many unique names logged into the authentication system?
  • What DNS servers are the kids using? Latvia?

Find the right log file and look through it.

I don\'t want to talk about graphing today, except to say that regardless of how I graph, I will need to produce “time series” data.

d o r k and other stuff

And other stuff . . .

DHCP protocol
ra ra ra

If you believe DORK, this should be RK.

Clients renew (or try, anyway) by sending a request to the server that last gave them a lease. If they get an ack, then they get to keep that address. If they get a nak, the address is not available. Or the client might get no response at all.

example how many mac addresses

Make a list of all the ethernet addresses in the transaction log. Eliminate the duplicates. Count what is left. Pretty simple.

We need some tools . . .

Example: How many MAC addresses?
unix tool grep

Grep applies a pattern to each line of a file. If the pattern exists in the line, the line is printed. If the pattern doesn\'t match, the line is skipped and the next line is considered.

Print can mean print or it can mean pass to the next program in a chain.

Unix tool: grep
unix tool awk

Awk also operates on lines. It can reference the words within the line, testing them for a match, doing arithmetic on numbers, or just printing them.

Awk uses curly braces { } to surround its arguments.

Unix tool: awk
finding the mac address

Oct 10 00:01:34 dhcpa dhcpd: [ID 702911 local7.info] DHCPREQUEST for from d4:9a:20:a5:32:ab (iPod-touch-3) via

Oct 10 00:01:34 dhcpa dhcpd: [ID 702911 local7.info] DHCPREQUEST for ( from 00:13:72:bc:f5:81 (se81) via

In both cases, the address follows the word \'from\'

Finding the MAC address
example w grep and awk

grep DHCPREQUEST dhcploga | awk \'{if ($12 == "from") print $13; if ($13 == "from") print $14}\'

Can also be written as:

grep DHCPREQUEST dhcploga | \

awk \'{if ($12 == "from") print $13; \

if ($13 == "from") print $14}\'

Example w grep and awk
output looks like









On Friday, there were 280K lines on dhcpa

Output looks like:
unix tool uniq

Think unique

This tool compares lines in a file. When two or more lines are identical, it replaces them with a single line. But for this to be useful, all the addresses that are the same need to be adjacent in the file. Said another way, the file needs to be sorted.

Unix tool: uniq
unix tool sort

sort has lots of options. It can sort numerically or alphabetically. It can start sorting on a column other than the first word. It can reverse sort. Here, it doesn\'t make any difference because we only need all occurrences of the same MAC address to be on adjacent lines.

Unix tool: sort
putting this together we get

grep DHCPREQUEST dhcploga | \

awk \'{if ($12 == "from") print $13; \

if ($13 == "from") print $14}\' | \

sort | uniq | wc

wc counts the lines and words in the file. The number of lines is our answer: the number of different MAC addresses seen by the system.

Putting this together, we get . . .
answers for last week
Answers for last week
  • Friday 17391
  • Thursday 19451
  • Wednesday 19435
  • Tuesday 19668
  • Monday 19040
anything wrong here
Anything wrong here?
  • There are two servers – dhcpa and dhcpb. I should have combined their logs.
  • DHCP assigns addresses to interfaces, not computers. One computer that is on wireless and wired at the same time will have two addresses.
  • A computer that is used on resnet (wired) in the dorm and cruznet (wireless) on campus will show us both MAC addresses, even if they\'re not used at the same time.
another example iphone count
Another example: iPhone count
  • There is a hint how this might be done in the log file slide.
  • If we collect all the Client-hostnames, sort and run uniq we will not get the right answer.
  • Ethernet MAC addresses are unique. But is no reason to assume Client-hostnames are unique. Monicas-iPhone? Could be lots of Monicas.
  • How could we fix this?
client hostname

For MAC address we had:

grep DHCPREQUEST dhcploga | \

awk \'{if ($12 == "from") print $13; \

if ($13 == "from") print $14}\'

Change this to:

grep DHCPREQUEST dhcploga | \

awk \'{if ($12 == "from") print $14, $13; \

if ($13 == "from") print $15, $14}\'

this looks like

(Austin1) 00:26:bb:0d:26:54

(Austins-iPhone) cc:08:e0:1e:3b:43

(Austins-phone) 00:26:08:7c:87:fa

(Autumn-Saga) 00:25:bc:e1:5e:98

(Avery) 00:26:4a:18:da:7a

(Aviel-PC) 00:1b:24:f6:9d:0a

(Aviv-PC) f0:7b:cb:0c:04:fb

(AwesomeIV) f8:1e:df:d8:90:95

(AwesomePossum) 00:16:d4:a5:af:6c

(Awesomeness-PC) 00:23:4d:73:72:8c

This looks like . . .
phone strategy
Phone strategy
  • Select client names that contain phone with grep -i
  • -i option ignores case so we get both phone and Phone
  • Retain the MAC addresses of the phones, and look for unique ones as before, so

[stuff from previous page] | grep -i phone | \

awk \'{print $2}\' | sort | uniq | wc

apple counts
Apple counts:
  • 399 iPhones
  • 552 iPods
  • 42 iPads

How do we know this is right?

We can check the MAC vendor code to make sure it is Apple

But really, we don\'t know.

dhcp leases with awk

The DHCP server keeps a record of current leases in a file on the server disk. If the server needs to restart, the leases file tells it about leases already granted. There is one record for each managed IP address. Each address is in one of several states:

DHCP leases with awk
dhcp lease states an ip address can be
DHCP lease statesAn IP-address can be:
  • FREE – address is available for assignment
  • BACKUP – address is free but on other server
  • ACTIVE – unexpired lease held by client
  • RELEASED – half way toward FREE
  • ABANDONED – address conflict

An address is put into abandoned status when a client trys to use it and finds that it is already in use. ABANDONED is the same as FREE except that the address will not be offered if untainted free addresses are available.

a leases entry looks like

Lease {

Starts 3 2010/10/06 02:29:38;

Ends 3 2010/10/06 05:15:00;

Tstp 3 2010/10/06 05:15:00;

Tsfp 3 2010/10/06 17:32:08;

Cltt 3 2010/10/6 02:29:28;

Binding state released;

Next binding state free;

Hardware ethernet a4:ba:db:b3:5c:cc;

Client-hostname “Catalina-PC”;


A leases entry looks like:
lease entry con t

There may be other lines in the file, depending on the client. For example, dhcp option 82 might give switch/port information.

There is a block like this for every dynamic address the server is managing.

Lease entry (con\'t)
writing to the leases file

When an address changes state, a new entry for that address is written to the end of the leases file. For any address, the only entry that is current is the LAST one. The server never changes entries in the middle of the file.

Writing to the leases file
  • Each address lease block is 10+ lines. Since grep acts on lines individually, it won\'t work to select whole blocks.
  • It would sure be nice if each address was one line.
  • awk commands could get to be pretty long and hard to type. Commands can be placed in a file. This file might be called an awk script. Then it becomes:

awk -f script.awk

script to fix the multi line problem



# Input is dhcp.leases. Output is one line per addr.


if ($1 == "lease") printf "\n%15s ",$2

if ($1 == "starts") printf "%s %s ",$3, $4

if ($1 == "ends") printf "%s %s ", $3, $4

if ($1 == "binding") printf "%8s ", $3

if ($1 == "hardware") printf "%s ",$3

if ($1 == "uid") printf "%s ", $2

if ($1 == "client-hostname") printf "%s ", $2


Script to fix the multi-line problem
this produces long lines like 2010/10/11 02:32:42; 2010/10/11 14:32:42; active; 00:22:fa:df:6d:c8; "\001\000\"\372\337m\310"; "Kunal-PC"; 2009/07/21 12:16:06; 2009/04/09 15:54:05; backup; 00:23:6c:05:8a:4f; "\001\000#l\005\212O";

The UID is weird binary. Time is GMT. Client-ID is optional for the client.

This produces long lines like:
usually interest in one subnet

We\'ll use the 2-net as an example

awk -f script.awk dhcp.leases | \

awk -F. \'{if ($3 == 2) print}\'

-F. means to use the period as the word separator instead of white space. The line begins with an address like so the third word is the number 2.

Usually, interest in one subnet
# script name is lastline.awk


line[$1]= $0

} END {

for (item in line) print line[item]


$0 is the whole line. $1 is the IP address.

For each IP, keep only the last line
awk f script awk dhcp leases grep 128 114 awk f if 3 2 print awk f lastline awk grep active wc
awk -f script.awk dhcp.leases | grep 128.114 \

awk -F. \'{if ($3 == 2) print}\' | \

awk -f lastline.awk | grep active | wc

Count of leases in use on 2-net