File: //usr/share/doc/varnish/html/reference/varnishncsa.html
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>varnishncsa — Varnish version 6.4.0 documentation</title>
<link rel="stylesheet" href="../_static/default.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
VERSION: '6.4.0',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="../_static/jquery.js"></script>
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="top" title="Varnish version 6.4.0 documentation" href="../index.html" />
<link rel="up" title="The Varnish Reference Manual" href="index.html" />
<link rel="next" title="varnishstat" href="varnishstat.html" />
<link rel="prev" title="varnishlog" href="varnishlog.html" />
</head>
<body>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="varnishstat.html" title="varnishstat"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="varnishlog.html" title="varnishlog"
accesskey="P">previous</a> |</li>
<li><a href="../index.html">Varnish version 6.4.0 documentation</a> »</li>
<li><a href="index.html" accesskey="U">The Varnish Reference Manual</a> »</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="varnishncsa">
<span id="varnishncsa-1"></span><h1>varnishncsa<a class="headerlink" href="#varnishncsa" title="Permalink to this headline">¶</a></h1>
<div class="section" id="display-varnish-logs-in-apache-ncsa-combined-log-format">
<h2>Display Varnish logs in Apache / NCSA combined log format<a class="headerlink" href="#display-varnish-logs-in-apache-ncsa-combined-log-format" title="Permalink to this headline">¶</a></h2>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name" />
<col class="field-body" />
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">Manual section:</th><td class="field-body">1</td>
</tr>
</tbody>
</table>
<div class="section" id="synopsis">
<h3>SYNOPSIS<a class="headerlink" href="#synopsis" title="Permalink to this headline">¶</a></h3>
<p>varnishncsa [-a] [-b] [-c] [-C] [-d] [-D] [-E] [-F <format>] [-f <formatfile>] [-g <request|vxid>] [-h] [-L <limit>] [-n <dir>] [-P <file>] [-Q <file>] [-q <query>] [-r <filename>] [-R <limit[/duration]>] [-t <seconds|off>] [-V] [-w <filename>]</p>
</div>
<div class="section" id="description">
<h3>DESCRIPTION<a class="headerlink" href="#description" title="Permalink to this headline">¶</a></h3>
<p>The varnishncsa utility reads varnishd(1) shared memory logs and
presents them in the Apache / NCSA "combined" log format.</p>
<p>Each log line produced is based on a single Request type transaction
gathered from the shared memory log. The Request transaction is then
scanned for the relevant parts in order to output one log line. To
filter the log lines produced, use the query language to select the
applicable transactions. Non-request transactions are ignored.</p>
<p>The following options are available:</p>
<table class="docutils option-list" frame="void" rules="none">
<col class="option" />
<col class="description" />
<tbody valign="top">
<tr><td class="option-group">
<kbd><span class="option">-a</span></kbd></td>
<td>When writing output to a file, append to it rather than overwrite it.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">-b</span></kbd></td>
<td>Log backend requests. If -c is not specified, then only backend requests will trigger log lines.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">-c</span></kbd></td>
<td>Log client requests. This is the default. If -b is specified, then -c is needed to also log client requests</td></tr>
<tr><td class="option-group">
<kbd><span class="option">-C</span></kbd></td>
<td>Do all regular expression and string matching caseless.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">-d</span></kbd></td>
<td>Process log records at the head of the log and exit.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">-D</span></kbd></td>
<td>Daemonize.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">-E</span></kbd></td>
<td>Show ESI request.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">-F <var><format></var></span></kbd></td>
<td>Set the output log format string.</td></tr>
<tr><td class="option-group" colspan="2">
<kbd><span class="option">-f <var><formatfile></var></span></kbd></td>
</tr>
<tr><td> </td><td>Read output format from a file. Will read a single line from the specified file, and use that line as the format.</td></tr>
<tr><td class="option-group" colspan="2">
<kbd><span class="option">-g <var><request|vxid></var></span></kbd></td>
</tr>
<tr><td> </td><td>The grouping of the log records. The default is to group by vxid.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">-h</span></kbd></td>
<td>Print program usage and exit</td></tr>
<tr><td class="option-group">
<kbd><span class="option">-L <var><limit></var></span></kbd></td>
<td>Sets the upper limit of incomplete transactions kept before the oldest transaction is force completed. A warning record is synthesized when this happens. This setting keeps an upper bound on the memory usage of running queries. Defaults to 1000 transactions.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">-n <var><dir></var></span></kbd></td>
<td>Specify the varnishd working directory (also known as instance name) to get logs from. If -n is not specified, the host name is used.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">-P <var><file></var></span></kbd></td>
<td>Write the process' PID to the specified file.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">-Q <var><file></var></span></kbd></td>
<td>Specifies the file containing the VSL query to use. When multiple -Q or -q options are specified, all queries are considered as if the 'or' operator was used to combine them.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">-q <var><query></var></span></kbd></td>
<td>Specifies the VSL query to use. When multiple -q or -Q options are specified, all queries are considered as if the 'or' operator was used to combine them.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">-r <var><filename></var></span></kbd></td>
<td>Read log in binary file format from this file. The file can be created with <tt class="docutils literal"><span class="pre">varnishlog</span> <span class="pre">-w</span> <span class="pre">filename</span></tt>.</td></tr>
<tr><td class="option-group" colspan="2">
<kbd><span class="option">-R <var><limit[/duration]></var></span></kbd></td>
</tr>
<tr><td> </td><td>Restrict the output to the specified limit. Transactions exceeding the limit will be suppressed. The limit is specified as the maximum number of transactions (with respect to the chosen grouping method) and an optional time period. If no duration is specified, a default of <tt class="docutils literal"><span class="pre">s</span></tt> is used. The duration field can be formatted as in VCL (e.g. <tt class="docutils literal"><span class="pre">-R</span> <span class="pre">10/2m</span></tt>) or as a simple time period without the prefix (e.g. <tt class="docutils literal"><span class="pre">-R</span> <span class="pre">5/m</span></tt>). When in <tt class="docutils literal"><span class="pre">-g</span> <span class="pre">raw</span></tt> grouping mode, this setting can not be used alongside <tt class="docutils literal"><span class="pre">-i</span></tt>, <tt class="docutils literal"><span class="pre">-I</span></tt>, <tt class="docutils literal"><span class="pre">-x</span></tt> or <tt class="docutils literal"><span class="pre">-X</span></tt>, and we advise using <tt class="docutils literal"><span class="pre">-q</span></tt> instead.</td></tr>
<tr><td class="option-group" colspan="2">
<kbd><span class="option">-t <var><seconds|off></var></span></kbd></td>
</tr>
<tr><td> </td><td>Timeout before returning error on initial VSM connection. If set the VSM connection is retried every 0.5 seconds for this many seconds. If zero the connection is attempted only once and will fail immediately if unsuccessful. If set to "off", the connection will not fail, allowing the utility to start and wait indefinetely for the Varnish instance to appear. Defaults to 5 seconds.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">-V</span></kbd></td>
<td>Print version information and exit.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">-w <var><filename></var></span></kbd></td>
<td>Redirect output to file. The file will be overwritten unless the -a option was specified. If the application receives a SIGHUP in daemon mode the file will be reopened allowing the old one to be rotated away. This option is required when running in daemon mode.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">--optstring</span></kbd></td>
<td>Print the optstring parameter to <tt class="docutils literal"><span class="pre">getopt(3)</span></tt> to help writing wrapper scripts.</td></tr>
</tbody>
</table>
</div>
<div class="section" id="modes">
<h3>MODES<a class="headerlink" href="#modes" title="Permalink to this headline">¶</a></h3>
<p>The default mode of varnishncsa is "client mode". In this mode, the
log will be similar to what a web server would produce in the absence
of varnish. Client mode can be explicitly selected by using -c.</p>
<p>If the -b switch is specified, varnishncsa will operate in "backend
mode". In this mode, requests generated by varnish to the backends
will be logged. Unless -c is also specified, client requests received
by varnish will be ignored.</p>
<p>When running varnishncsa in both backend and client mode, it is
strongly advised to include the format specifier %{Varnish:side}x to
distinguish between backend and client requests.</p>
<p>Client requests that results in a pipe (ie. return(pipe) in vcl), will
not generate logging in backend mode. This is because varnish is not
generating requests, but blindly passes on bytes in both directions.
However, a varnishncsa instance running in normal mode can see this
case by using the formatter %{Varnish:handling}x, which will be 'pipe'.</p>
<p>In backend mode, some of the fields in the format string get different
meanings. Most notably, the byte counting formatters (%b, %I, %O)
considers varnish to be the client.</p>
<p>It is possible to keep two varnishncsa instances running, one in
backend mode, and one in client mode, logging to different files.</p>
</div>
<div class="section" id="format">
<h3>FORMAT<a class="headerlink" href="#format" title="Permalink to this headline">¶</a></h3>
<p>Specify the log format to use. If no format is specified the default log
format is used:</p>
<div class="highlight-python"><pre>%h %l %u %t "%r" %s %b "%{Referer}i" "%{User-agent}i"</pre>
</div>
<p>Escape sequences \n and \t are supported.</p>
<p>Supported formatters are:</p>
<dl class="docutils">
<dt>%b</dt>
<dd>In client mode, size of response in bytes, excluding HTTP headers.
In backend mode, the number of bytes received from the backend,
excluding HTTP headers. In CLF format, i.e. a '-' rather than a 0
when no bytes are sent.</dd>
<dt>%D</dt>
<dd>In client mode, time taken to serve the request, in microseconds.
In backend mode, time from the request was sent to the entire body
had been received.</dd>
<dt>%H</dt>
<dd>The request protocol. Defaults to HTTP/1.0 if not known.</dd>
<dt>%h</dt>
<dd>Remote host. Defaults to '-' if not known. In backend mode this is
the IP of the backend server.</dd>
<dt>%I</dt>
<dd>In client mode, total bytes received from client. In backend mode,
total bytes sent to the backend.</dd>
<dt>%{X}i</dt>
<dd>The contents of request header X. If the header appears multiple times
in a single transaction, the last occurrence is used.</dd>
<dt>%l</dt>
<dd>Remote logname. Always '-'.</dd>
<dt>%m</dt>
<dd>Request method. Defaults to '-' if not known.</dd>
<dt>%{X}o</dt>
<dd>The contents of response header X. If the header appears multiple
times in a single transaction, the last occurrence is used.</dd>
<dt>%O</dt>
<dd>In client mode, total bytes sent to client. In backend mode, total
bytes received from the backend.</dd>
<dt>%q</dt>
<dd>The query string. Defaults to an empty string if not present.</dd>
<dt>%r</dt>
<dd>The first line of the request. Synthesized from other fields, so it
may not be the request verbatim. See the NOTES section.</dd>
<dt>%s</dt>
<dd>Status sent to the client. In backend mode, status received from
the backend.</dd>
<dt>%t</dt>
<dd>In client mode, time when the request was received, in HTTP
date/time format. In backend mode, time when the request was sent.</dd>
<dt>%{X}t</dt>
<dd>In client mode, time when the request was received, in the format
specified by X. In backend mode, time when the request was sent.
The time specification format is the same as for strftime(3).</dd>
<dt>%T</dt>
<dd>In client mode, time taken to serve the request, in seconds. In
backend mode, time from the request was sent to the entire body had
been received.</dd>
<dt>%U</dt>
<dd>The request URL without the query string. Defaults to '-' if not
known.</dd>
<dt>%u</dt>
<dd>Remote user from auth.</dd>
<dt>%{X}x</dt>
<dd><p class="first">Extended variables. Supported variables are:</p>
<dl class="last docutils">
<dt>Varnish:time_firstbyte</dt>
<dd>Time from when the request processing starts until the first
byte is sent to the client, in seconds. For backend mode: Time
from the request was sent to the backend to the entire header
had been received.</dd>
<dt>Varnish:hitmiss</dt>
<dd>One of the 'hit' or 'miss' strings, depending on whether the request
was a cache hit or miss. Pipe, pass and synth are considered misses.</dd>
<dt>Varnish:handling</dt>
<dd>One of the 'hit', 'miss', 'pass', 'pipe' or 'synth' strings indicating
how the request was handled.</dd>
<dt>Varnish:side</dt>
<dd>Backend or client side. One of two values, 'b' or 'c', depending
on where the request was made. In pure backend or client mode,
this field will be constant.</dd>
<dt>Varnish:vxid</dt>
<dd>The VXID of the varnish transaction.</dd>
<dt>VCL_Log:key</dt>
<dd>The value set by std.log("key:value") in VCL.</dd>
<dt>VSL:tag:record-prefix[field]</dt>
<dd><p class="first">The value of the VSL entry for the given tag-record prefix-field
combination. Tag is mandatory, the other components are optional.</p>
<p>The record prefix will limit the matches to those records that
have this prefix as the first part of the record content followed
by a colon.</p>
<p>The field will, if present, treat the log record as a white
space separated list of fields, and only the nth part of the
record will be matched against. Fields start counting at 1 and
run up to 255.</p>
<p class="last">Defaults to '-' when the tag is not seen, the record prefix
does not match or the field is out of bounds. If a tag appears
multiple times in a single transaction, the first occurrence
is used.</p>
</dd>
</dl>
</dd>
</dl>
</div>
<div class="section" id="signals">
<h3>SIGNALS<a class="headerlink" href="#signals" title="Permalink to this headline">¶</a></h3>
<ul>
<li><p class="first">SIGHUP</p>
<p>Rotate the log file (see -w option) in daemon mode,
abort the loop and die gracefully when running in the foreground.</p>
</li>
<li><p class="first">SIGUSR1</p>
<p>Flush any outstanding transactions.</p>
</li>
</ul>
</div>
<div class="section" id="notes">
<h3>NOTES<a class="headerlink" href="#notes" title="Permalink to this headline">¶</a></h3>
<p>The %r formatter is equivalent to "%m <a class="reference external" href="http:/">http:/</a>/%{Host}i%U%q %H". This
differs from apache's %r behavior, equivalent to "%m %U%q %H".
Furthermore, when using the %r formatter, if the Host header appears
multiple times in a single transaction, the first occurrence is used.</p>
</div>
<div class="section" id="example">
<h3>EXAMPLE<a class="headerlink" href="#example" title="Permalink to this headline">¶</a></h3>
<p>Log the second field of the Begin record, corresponding to the VXID
of the parent transaction:</p>
<div class="highlight-python"><pre>varnishncsa -F "%{VSL:Begin[2]}x"</pre>
</div>
<p>Log the entire Timestamp record associated with the processing length:</p>
<div class="highlight-python"><pre>varnishncsa -F "%{VSL:Timestamp:Process}x"</pre>
</div>
</div>
<div class="section" id="see-also">
<h3>SEE ALSO<a class="headerlink" href="#see-also" title="Permalink to this headline">¶</a></h3>
<p><a class="reference internal" href="varnishd.html#varnishd-1"><em>varnishd</em></a>
<a class="reference internal" href="varnishlog.html#varnishlog-1"><em>varnishlog</em></a>
<a class="reference internal" href="varnishstat.html#varnishstat-1"><em>varnishstat</em></a>
<a class="reference internal" href="vsl.html#vsl-7"><em>VSL</em></a></p>
</div>
<div class="section" id="history">
<h3>HISTORY<a class="headerlink" href="#history" title="Permalink to this headline">¶</a></h3>
<p>The varnishncsa utility was developed by Poul-Henning Kamp in
cooperation with Verdens Gang AS and Varnish Software AS. This manual page was
initially written by Dag-Erling Smørgrav <<a class="reference external" href="mailto:des%40des.no">des<span>@</span>des<span>.</span>no</a>>, and later updated
by Martin Blix Grydeland and Pål Hermunn Johansen.</p>
</div>
<div class="section" id="copyright">
<h3>COPYRIGHT<a class="headerlink" href="#copyright" title="Permalink to this headline">¶</a></h3>
<p>This document is licensed under the same licence as Varnish
itself. See LICENCE for details.</p>
<ul class="simple">
<li>Copyright (c) 2006 Verdens Gang AS</li>
<li>Copyright (c) 2006-2016 Varnish Software AS</li>
</ul>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar">
<div class="sphinxsidebarwrapper">
<h3><a href="../index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">varnishncsa</a><ul>
<li><a class="reference internal" href="#display-varnish-logs-in-apache-ncsa-combined-log-format">Display Varnish logs in Apache / NCSA combined log format</a><ul>
<li><a class="reference internal" href="#synopsis">SYNOPSIS</a></li>
<li><a class="reference internal" href="#description">DESCRIPTION</a></li>
<li><a class="reference internal" href="#modes">MODES</a></li>
<li><a class="reference internal" href="#format">FORMAT</a></li>
<li><a class="reference internal" href="#signals">SIGNALS</a></li>
<li><a class="reference internal" href="#notes">NOTES</a></li>
<li><a class="reference internal" href="#example">EXAMPLE</a></li>
<li><a class="reference internal" href="#see-also">SEE ALSO</a></li>
<li><a class="reference internal" href="#history">HISTORY</a></li>
<li><a class="reference internal" href="#copyright">COPYRIGHT</a></li>
</ul>
</li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="varnishlog.html"
title="previous chapter">varnishlog</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="varnishstat.html"
title="next chapter">varnishstat</a></p>
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="../_sources/reference/varnishncsa.txt"
rel="nofollow">Show Source</a></li>
</ul>
<div id="searchbox" style="display: none">
<h3>Quick search</h3>
<form class="search" action="../search.html" method="get">
<input type="text" name="q" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="varnishstat.html" title="varnishstat"
>next</a> |</li>
<li class="right" >
<a href="varnishlog.html" title="varnishlog"
>previous</a> |</li>
<li><a href="../index.html">Varnish version 6.4.0 documentation</a> »</li>
<li><a href="index.html" >The Varnish Reference Manual</a> »</li>
</ul>
</div>
<div class="footer">
© Copyright 2010-2014, Varnish Software AS.
Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.1.3.
</div>
</body>
</html>