Apache quick stats

When troubleshooting performance issues with Apache that happened earlier in the day or week, it is very useful to parse the logs quickly to determine quick facts about the inbound connection rates. It may reveal a period of increased traffic that needs to be investigated further, therefore giving you that thread to begin unraveling the problem.

To get the connections per hour for a specific day, run the following:

[[email protected] ~]# grep "27/Feb" /var/log/httpd/www.example.com-access.log | cut -d[ -f2 | cut -d] -f1 | awk -F: '{print $2":00"}' | sort -n | uniq -c
  50205 03:00
  90516 04:00
  64837 05:00
  47410 06:00
  44876 07:00
  41098 08:00
  38996 09:00
  37234 10:00
  43704 11:00
  58702 12:00
  58922 13:00
  72592 14:00
  77792 15:00
  88882 16:00
  80815 17:00
  98287 18:00
 617857 19:00
  90507 20:00
  98568 21:00
 147584 22:00
 181814 23:00

Based off that output, there was a massive spike in connections during the 9:00PM hour (19:00). So now lets break the 9:00PM hour down to show the connections per minute:

[[email protected] ~]# grep "27/Feb/2017:19" /var/log/httpd/www.example.com-access.log | cut -d[ -f2 | cut -d] -f1 | awk -F: '{print $2":"$3}' | sort -nk1 -nk2 | uniq -c | awk '{ if ($1 > 10) print $0}'
   1629 19:00
   1664 19:01
   1840 19:02
  10493 19:03
  13728 19:04
  17608 19:05
   1377 19:06
   2333 19:07
   1980 19:08
   2056 19:09
   2123 19:10
...
   1997 19:57
   1631 19:58
   1988 19:59

As shown above, there was some sort of traffic spike that occurred between 9:03PM – 9:05PM. As the window has been narrowed down to a 3 minute period, more specific analysis can be performed. The examples below will focus on what was happening around 9:03PM.

To list the top 10 IP’s accessing the site during around 9:03PM

[[email protected] ~]# grep "27/Feb/2017:19:03" /var/log/httpd/www.example.com-access.log | awk '{print $1}' | sort -nr | uniq -c |sort -nr | head

To list the top most called elements on the site:

[[email protected] ~]# grep "27/Feb/2017:19:03" /var/log/httpd/www.example.com-access.log | awk '{print $7}' | sort -nr | uniq -c | sort -nr | head