In order to effectively manage a web server, it is necessary to get feedback about the activity and performance of the server as well as any problems that may be occurring. The HTTPD Logs can provide everything that happens on your server, from the initial request, through the URL mapping process, to the final resolution of the connection, including any errors that may have occurred in the process.
Every time your server receives a request, it makes an entry to one or more log files. These log files are useful for a variety of purposes, from statistical analysis of your visitors to forensic analysis of an attack on your server. The HTTPD Logs sample is intended to show the capability of WSO2 BAM which can analyze the raw httpd logs and produce useful result. This sample demonstrate how you can use your logs to analyse the web traffic come to your server from different regions. It calculate the region from the ip address in logs and visualize it in gadgets.
1. Start the WSO2 BAM Server.
2. Go to $WSO2_BAM_HOME/samples/httpd-logs directory via console
3. Type 'ant' from the console (This will read the access.log from the $WSO2_BAM_HOME/samples/httpd-logs/resources Directory and send each log line as event)
Note that you can give custom values for parameters for Data receiver URL, user name and password. Default values of them are as follows.
Now lets see how new values can be given as parameters
Command with new values : ant -Durl=tcp://localhost:7612 -Dusername=user -Dpassword=pass123
Part of the sample log file:
Published event format (Stream definition)
A sample event
4. Go to Management console of WSO2 BAM.
5. Go to Main -> BAM ToolBox -> Add. Select "HTTPD Logs Analysis Toolbox" and click on Install button.
6. Go to Main -> BAM Toolbox -> List. Wait until the 'HTTPD Logs Analysis Toolbox' toolbox status changes to Installed.
7. Wait for some time, until script complete the first run after publishing the data. (Script will run in each minute).
If hive job is successful you will see the following message on BAM console.
Ended Job = job_local_0001
Execution completed successfully
Mapred Local Task Succeeded . Convert the Join into MapJoin
8. Click on Main -> Gadgets -> View portal to see the populated gadgets which shows a summary of no of requests coming from different countries.