Friday, October 31, 2008

Analyzing IIS log files

AWStats is an open source free tool that can be used to analyze IIS log files. It can be downloaded from http://awstats.sourceforge.net/

Setup

  1. Perl is a prerequisite for AWStats. A version of this can be downloaded from http://www.activestate.com/Products/activeperl/index.mhtml. This tool also adds perl command to PATH environment variable
  2. Download AWStats from http://awstats.sourceforge.net/#DOWNLOAD
  3. Install AWStats to your hard disk (say D:\Program Files\AWStats)
  4. Create a folder for AWStats on your hard disk. It will be used as web deployment folder for AWStats (say C:\Inetpub\wwwroot\awstats)
  5. Copy D:\Program Files\AWStats\wwwroot\*.* to C:\Inetpub\wwwroot\awstats\
  6. Make a copy of C:\Inetpub\wwwroot\awstats\cgi-bin\awstats.model.conf and name it as C:\Inetpub\wwwroot\awstats\cgi-bin\awstats.mysite.conf where mysite is domain of the website (it can be anything though)
  7. Edit awstats.mysite.conf and set following properties
  • LogFile="D:\Production Log\39\ex081030.log" (path to log file being analyzed)
  • LogFormat="date time s-sitename s-computername s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs-version cs(User-Agent) cs(Cookie) cs(Referer) cs-host sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken" (This format can be copied from IIS log file. Open log file in text editor and copy from this line at top #Fields: date time s-sitename s-computername s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs-version cs(User-Agent) cs(Cookie) cs(Referer) cs-host sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken
  • SiteDomain=www.mysite.com
  • There are other properties which can be customized. Refer to doc in D:\Program Files\AWStats\docs
  1. On command prompt enter following commands

    perl awstats.pl -config=mysite -update
    perl awstats.pl -config=mysite -output -staticlinks > awstats.mysite.html
    perl awstats.pl -config=mysite -output=alldomains -staticlinks > awstats.mysite.alldomains.html
    perl awstats.pl -config=mysite -output=allhosts -staticlinks > awstats.mysite.allhosts.html
    perl awstats.pl -config=mysite -output=lasthosts -staticlinks > awstats.mysite.lasthosts.html
    perl awstats.pl -config=mysite -output=unknownip -staticlinks > awstats.mysite.unknownip.html
    perl awstats.pl -config=mysite -output=alllogins -staticlinks > awstats.mysite.alllogins.html
    perl awstats.pl -config=mysite -output=lastlogins -staticlinks > awstats.mysite.lastlogins.html
    perl awstats.pl -config=mysite -output=allrobots -staticlinks > awstats.mysite.allrobots.html
    perl awstats.pl -config=mysite -output=lastrobots -staticlinks > awstats.mysite.lastrobots.html
    perl awstats.pl -config=mysite -output=urldetail -staticlinks > awstats.mysite.urldetail.html
    perl awstats.pl -config=mysite -output=urlentry -staticlinks > awstats.mysite.urlentry.html
    perl awstats.pl -config=mysite -output=urlexit -staticlinks > awstats.mysite.urlexit.html
    perl awstats.pl -config=mysite -output=browserdetail -staticlinks > awstats.mysite.browserdetail.html
    perl awstats.pl -config=mysite -output=osdetail -staticlinks > awstats.mysite.osdetail.html
    perl awstats.pl -config=mysite -output=unknownbrowser -staticlinks > awstats.mysite.unknownbrowser.html
    perl awstats.pl -config=mysite -output=unknownos -staticlinks > awstats.mysite.unknownos.html
    perl awstats.pl -config=mysite -output=refererse -staticlinks > awstats.mysite.refererse.html
    perl awstats.pl -config=mysite -output=refererpages -staticlinks > awstats.mysite.refererpages.html
    perl awstats.pl -config=mysite -output=keyphrases -staticlinks > awstats.mysite.keyphrases.html
    perl awstats.pl -config=mysite -output=keywords -staticlinks > awstats.mysite.keywords.html
    perl awstats.pl -config=mysite -output=errors404 -staticlinks > awstats.mysite.errors404.html


    I prefer to create a batch file with all these commands and pass site name as parameter.

    Once executed, it will create multiple .html files in C:\Inetpub\wwwroot\awstats\cgi-bin (provided you have not configured any other path in awstats.mysite.conf). Open http://localhost/.../awstats.mysite.html (depends on where you created web deployment folder) which is the index page and you are done.
    Hope this helps when you have to analyze log files next time :)