wflogs - firewall log analyser of the WallFire project.
SYNOPSIS
wflogs [options] [logfile]
DESCRIPTION
wflogs
is a firewall log analyser. It can be used to produce a log summary report
in plain text, HTML and XML, or even to translate a log file into another
firewall log format, for example.
Logs can be filtered, summarized, sorted, and obfuscated (in that order),
using the following options.
By default, output is not sorted, and may be summarized if the output
module has a `summary' option and if this option is set to `yes' (even
by default value).
You have to specify a module name that will handle the input (parsing)
and another for the output (exportation). See MODULES sections below.
With no logfile, wflogs read /var/log/messages.
When logfile is `-', it reads standard input.
OPTIONS
-c | --config file
wflogs will use given configuration file. If not specified, wflogs
will not use any configuration file and will only use command line options.
-f | --filter expression
Print log entries that match the boolean expression.
This expression looks very much like a Perl condition, which must
be passed as a single, quoted argument.
If no expression is given, all log entries will be dumped.
Otherwise, only entries for which expression is `true' will be dumped.
See the FILTER EXPRESSION section below.
-i | --input-format format[,format2,...]
Specify the input parsing modules. Wflogs will use the
corresponding modules (if available) to parse the logs.
If you want to parse a log file with multiple formats mixed (typically
a remote syslog file), you can specify several format module names
separated by commas, one being probed after another.
Use special name `all' to try every available format.
If you omit the `-i' option, wflogs will try to guess the local
firewalling tool at runtime, and use the corresponding module.
Use format `help' to know which modules are available (currently,
`netfilter', `ipchains', `ipfilter', `cisco_pix', `cisco_ios', and `snort'),
and which is the default (guessed) module.
See INPUT MODULES section below.
-I, --interactive
Interactive mode. The program will not terminate, but enter a little
interactive shell.
This option can be used in conjonction with real-time mode (-R option).
While in non-interactive real-time mode (-R only), signal USR1 enables
to fall back into interactive mode.
-o | --output-type type [ output module options ]
Specify the output module type. Wflogs will use the
corresponding module (if available) to export the input logs to the
corresponding target.
Use type `help' to know which modules are available (currently,
`text', `html', `xml', `human', `netfilter', `ipchains', and `ipfilter').
Default mode is `text'.
See OUTPUT MODULES section below.
Output module configuration can be achieved via the command line.
You can specify long options (with a `--' prefix). Three types are
supported: boolean (yes or no), integer, and string.
A special option `--options' displays the available options of the module,
with type, help message, and default value. For example,
wflogs -o html --options
shows the HTML output module configuration.
-O | --obfuscate [criterias]
This option obfuscates some logging fields according to given criterias,
separated by commas. These can be `date', `hostname', `ipaddr', or `macaddr'
(or `all' for everything). Default (if no criteria is given) is `all'.
If ipaddr is specified, output module options `resolve' and `whois_lookup'
(if available) are set to no. If macaddr is specified, output module
option `mac_vendor' (if available) is set to no.
Date order is conserved, hostnames are replaced by "hostx" (where
x is a growing number), ipaddr belong to 0.0.0.0/8 network, macaddr
are of the form 0:0:0:0:0:1, 0:0:0:0:0:2, etc. Note that for all obfuscated
fields, each original value is associated with a new unique one (unicity
is preserved).
-P | --proceed
If real-time (-R) or interactive (-I) modes are set, first process log
entries in the input logfile before entering in these modes, as these
entries won't be parsed by default in these modes.
-R | --realtime
``Real-time'' mode: logs are monitored in real-time. Wflogs will
wait for new log entries. Entries already present in the input logfile
will not be processed as usual, unless you specify -P option.
This option can be used in conjonction with interactive mode (-I option).
While in non-interactive real-time mode (-R only), signal USR1 enables
to fall back into interactive mode.
-s | --sort[=criteria_list]
Set output lines sort order according to the multilevel sort
specified by the sequence of keys key1,key2,... Syntax is
--sort=[+|-]key1[,[+|-]key2[,...]]. Choose a key from the SORT KEYS
section. `-' reverses direction only on the key it precedes. The `+'
is really optional since default direction is increasing numerical or
lexicographic order.
For example wflogs --sort=dport,-time sorts according to
destination port number, then reverse time (for a given port number).
If one of the keys is `none', the output is not sorted.
Use key `help' to show available keys. If no sort criteria is given,
output is sorted by with `-count,time,dipaddr,protocol,dport'.
--strict-parsing type
Set the parsing policy. Available types are: `loose' (even if there are
garbage in the input file or incorrect log lines, parse as much as possible
and issue no warning at all),
`nowarning' (in this case, issue no warnings, ignore non-log lines but
do not store incoherent entries), `warning' (issue warnings on stderr,
ignore non-log lines but do not store bizarre entries), and `error'
(stop parsing if line is not a log entry, or if entry is bizarre).
Default type is `warning'.
-v | --verbose [level]
Set verbosity level. If level is omitted, default value is 1.
-V | --version
Display current version.
-h | --help
Show help message on stdout.
INPUT MODULES
wflogs can use extended input modules, each one parsing a specific
firewall log format. See option -i.
netfilter
This module parse the
netfilter
log format.
ipchains
This module parse the
ipchains
log format.
ipfilter
This module parse the
ipfilter
log format.
cisco_pix
This module parse the
cisco PIX
and
cisco FWSM
log format.
cisco_ios
This module parse the
cisco IOS
log format.
snort
This module parse the
snort IDS
ACLs log format.
OUTPUT MODULES
wflogs can use extended output modules, which enable to export the
input logs to a particular format. So it can be used to rewrite the
input into another firewall log format or generate a report, for
example. See option -o. Summary mode depends on the module, and is
configurable through the `summary' module option.
text
This module produces a summary in text mode. Please note that this text
output is not intended to be parsed. Use XML output module instead.
html
This module produces a summary output in HTML format.
xml
This module produces a summary in XML format (see wflogs DTD).
human
This module produces a summary in text format, in a human readable form.
Newcomers may like it. ;-)
netfilter
This module exports input logs to
netfilter
log syntax.
ipchains
This module exports input logs to
ipchains
log syntax.
ipfilter
This module exports input logs to
ipfilter
log syntax.
SORT KEYS
KEY DESCRIPTION
count sort by count (number of original log entries)
time sort by log entry date (if count != 1, the date of the
first original log line)
timeend sort by log entry end date (if count != 1, the date of the
last original log line)
input_iface sort by input interface name
output_iface sort by output interface name
sipaddr sort by source IP address
dipaddr sort by destination IP address
smacaddr sort by source MAC address
dmacaddr sort by destination MAC address
protocol sort by protocol number
sport sort by source port number (if available)
dport sort by destination port number (if available)
tcpflags sort by TCP flags
hostname sort by hostname
chainlabel sort by chain label
branchname sort by branch name
datalen sort by data length
format sort by firewalling tool format
none do not sort
FILTER EXPRESSION
This filtering expression looks very much like a Perl condition.
Variables are prefixed with `$'.
Pre-defined variables are:
$format (string)
firewalling tool format
$count (integer)
number of original log entries
$start_time ([string] or integer)
log entry date (if count != 1, the date of the first original log line),
in date format ([string], see below), or in seconds since the Epoch
$end_time ([string] or integer)
log entry end date (if count != 1, the date of the last original log line),
in date format ([string], see below), or in seconds since the Epoch
$hostname (string)
name of the host which logged the packet
$chainlabel (string)
chain label
$branchname (string)
branch name
$input_iface (string)
input interface name
$output_iface (string)
output interface name
$protocol (integer)
protocol number (or name used in /etc/protocols)
$datalen (integer)
data length
$sipaddr (IP network)
source IP address, or source IP network
$sport (integer)
source port number (or name used in /etc/services) if protocol is UDP or TCP,
and ICMP type number or name if protocol is ICMP (this may change in the
future)
$smacaddr (MAC address)
source MAC address
$dipaddr (IP network)
destination IP address, or destination IP network
$dport (integer)
destination port number (or name used in /etc/services) if protocol is UDP or
TCP, and ICMP code number or name if protocol is ICMP (this may change in
the future)
$dmacaddr (MAC address)
destination MAC address
$tcpflags (integer)
TCP flags if protocol is TCP (flags can be a combination of
SYN|ACK|RST|FIN|PSH|URG|ECE|CWR)
For integer and boolean values, the following operators can be used:
||, &&, ==, !=, <, >, <=, >=,
&, |, ^, +, -.
String variables can be compared for strict equality with == and !=
operators, but also matched with an extended regular expression with =~
operator.
Strings are quoted with " (like "foo"), and regexps with /
(like /(foo|bar)/).
Note that regexp matches only a subset of the string. You have to
surround the regexp with ^ and $ if you want to match the whole
string (that may change in the future). Like in Perl, you may add an
optional i modifier after final /, to do case-insensitive pattern
matching.
Date format is one that is accepted by the getdate C function. It must be
enclosed in brackets [] and will be converted to an integer value which
stands for the number of seconds since the epoch (01 Jan 1970 UTC 00:00).
See DATE FORMAT section.
IP network can be an IP address, or an IP network (a.b.c.d/n.o.p.q
or a.b.c.d/bitmask, or even things like a.b.*.* for a /16 mask,
for example).
MAC addresses are of the form aa:bb:cc:dd:ee:ff. They can only be
compared for strict equality (== and != operators).
DATE FORMAT
The string may contain many flavors of items: calendar date items,
time of the day items, time zone items, day of the week item, relative items,
or pure numbers. As expression can be quite complex, if you have doubt
about the dates you specified, activate global verbose mode to show filter
expression on stderr using absolute dates.
Calendar date
can be "1974-08-31", "74-8-31", "74-08-31", "8/31/74",
"31 August 1974", "31 Aug 1974", "Aug 31, 1974", "31-aug-74", "31aug74".
The year can be omitted (current year is then used).
Time of day
can be "02:50:00", "02:50", "2:50am".
Day of week
can be "Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday" or
"Saturday", but can be abbreviated to their first three letters.
A number may precede a day of the week item to move forward
supplementary weeks. It is best used in expression like `third
monday'. In this context, `last DAY' or `next DAY' is also acceptable;
they move one week before or after the day that DAY by itself would
represent.
Relative items
adjust a date (or the current date if none) forward or backward.
It can be "1 year", "1 year ago", "3 years", "2 days", for example.
You can also use "month", "week", "day", "hour", "minute" ("min"),
and "second" ("sec"), or "now" ("today"), "yesterday", and "tomorrow".
The string `this' also has the meaning of a zero-valued time displacement,
but is preferred in date strings like `this thursday'.
Pure decimal number
precise intepretation depends on the context in the date string.
If the decimal number is of the form YYYYMMDD and no other calendar
date item appears before it in the date string, then YYYY is read as the
year, MM as the month number and DD as the day of the month, for the
specified calendar date.
If the decimal number is of the form HHMM and no other time of day
item appears before it in the date string, then HH is read as the hour
of the day and MM as the minute of the hour, for the specified time of
the day. MM can also be omitted.
EXAMPLES
wflogs -i netfilter -o html netfilter.log > logs.html
converts the given netfilter log file into a HTML report.
wflogs --sort=protocol,-time -i netfilter -o text netfilter.log > logs.txt
converts the given netfilter log file into a sorted (by protocol number,
then reverse time) text report.
wflogs -f '$start_time >= [this 3 days ago] && $start_time < [this 2 days ago] && $chainlabel =~ /(DROP|REJECT)/ && $sipaddr == 10.0.0.0/8 && $protocol == tcp && ($dport == ssh || $dport == telnet) && ($tcpflags & SYN)' -i netfilter -o text --summary=no
shows log entries (without summary) which match the given expression
(refused connection attempts that occured 3 days ago to ssh and telnet
ports coming from internal network 10.0.0.0/8).
wflogs -i netfilter --resolve=0 --whois=0 netfilter.log
converts the given netfilter log file into a text report (default mode),
disabling IP address reverse lookups and whois lookups.
wflogs -i netfilter -o xml netfilter.log > logs.xml
exports netfilter logs in XML.
wflogs -i ipchains -o netfilter ipchains.log > netfilter.log
converts ipchains logs into netfilter log format. So you may process them
with your favorite netfilter log analyser, for example (even if the latter
may not be better than wflogs itself. ;-)).
wflogs -i ipfilter -o human --datalen=yes ipfilter.log
produces a report about ipfilter logfile in natural language on stdout,
displaying packet length (datalen option) which is not showed by default.