pmlogreduce
reads one Performance Co-Pilot (PCP) archive
identified by
input
(this must be a PCP archive created by
pmlogger(1),
pmlogextract(1)
or
pmlogreduce(1)),
and creates a temporally reduced PCP archive in
output.
The
data reduction involves statistical and temporal reduction of samples with
an output sampling
interval defined by the
-t
option in the
output
archive (independent of the sampling intervals in the
input
archive), and is further controlled by
other command line arguments.
COMMAND LINE OPTIONS
The command line options for
pmlogreduce
are as follows:
-A align
Specify a ``natural'' alignment of the output sample times; refer
to
PCPIntro(1).
-S starttime
Define the start of a time window to restrict the samples retrieved
from the
input
archive; refer to
PCPIntro(1).
-s samples
The argument
samples
defines the number of samples to be written to
output.
If
samples
is 0 or
-s
is not specified,
pmlogreduce
will sample until the end of the PCP archive,
or the end of the time window as specified by
-T,
whichever comes first. The
-s
option will override the
-T
option if it occurs sooner.
-T endtime
Define the termination of a time window to restrict the samples
retrieved from the
input
archive; refer to
PCPIntro(1).
-v volsamples
The
output
archive is potentially a multi-volume data set, and the
-v
option causes
pmlogreduce
to start a new volume after
volsamples
log records have been written to the
output
archive.
-t interval
Consecutive samples in the
output
archive will appear with a time delta defined by
interval;
refer to
PCPIntro(1).
Note the default value is 600 (seconds, i.e. 10 minutes).
-Z timezone
Use
timezone
when displaying the date and time, or interpreting the
-S
and
-T
options.
Timezone
is in the format of the environment variable
TZ
as described in
environ(5).
-z
Use the local timezone of the host from the
input
archive when displaying the date and time, or interpreting the
-S
and
-T
options.
The default is to initially use the timezone of the local host.
DATA REDUCTION
The statistical and temporal reduction follows the following rules:
x.
Consecutive records from
input
are read without interpolation, and at most one output record
is written for each
interval,
summarizing the performance data over that period.
x.
If the semantics of a metric indicates it is
instantaneous
or
discrete
then
output
value is computed as the arithmetic mean of the observations (if any)
over each
interval.
x.
If the semantics of a metric indicates it is a
counter
then the following transformations are applied:
a)
Metrics with 32-bit precision are promoted to 64-bit precision.
b)
Any counter wrap (overflow) is noted, and appropriate adjustment made
in the value of the metric over each
interval.
This will be correct in the case of a single counter wrap, but will
silently
underestimate
in the case where more than one counter wrap occurs between consecutive
observations in the
input
archive, and silently
overestimate
in the case where a counter is reset occurs between consecutive
observations in the
input
archive; unfortunately these situations cannot be detected, but
are believed to be rare events for the sort of production monitoring
environments where
pmlogreduce
is most likely to be deployed.
x.
Any changes in instance domains, and indeed all metadata, is preserved.
x.
Any ``mark'' records in the
input
archive (as created by
pmlogextract(1))
will be preserved in the
output
archive, so periods where no data is available are maintained, and data
interpolation will
not
occur across these periods when the
output
archive is subsequently processed with PCP applications.
FILES
For each of the
input
and
output
archives, several physical files are used.
archive.meta
metadata (metric descriptions, instance domains, etc.) for the archive log
archive.0
initial volume of metrics values (subsequent volumes have suffixes
1
,
2,
...)
archive.index
temporal index to support rapid random access to the other files in the
archive log.
PCP ENVIRONMENT
Environment variables with the prefix
PCP_
are used to parameterize the file and directory names
used by PCP.
On each installation, the file
/etc/pcp.conf
contains the local values for these variables.
The
$PCP_CONF
variable may be used to specify an alternative
configuration file,
as described in
pcp.conf(4).
All error conditions detected by
pmlogreduce
are reported on
stderr
with textual (if sometimes terse) explanation.
Should the
input
archive be corrupted (this can happen
if the
pmlogger
instance writing the archive suddenly dies), then
pmlogreduce
will detect and report the position of the corruption in the file,
and any subsequent information from the
input
archive will not be processed.
If any error is detected,
pmlogreduce
will exit with a non-zero status.
CAVEATS
The preamble metrics (pmcd.pmlogger.archive, pmcd.pmlogger.host,
and pmcd.pmlogger.port), which are automatically recorded by
pmlogger
at the start of the archive, may not be present in the archive output by
pmlogreduce.
These metrics are only relevant while the archive is being created,
and have no significance once recording has finished.