[syslog-ng] Simple config question

[Miguel Alvarez]

Just wondering if anyone might have any idea as to why logs aren't
being forwarded.  I'm thinking maybe it has something to do with the
tagging configurations as I can definitely see the source log growing
(e.g. /var/log/snort/alert) and that when syslog-ng is started, it
successfully connects with the server side.  To me, it would seem then
that the problem lies somewhere in between.  The whole config is below
but here's a relevant snippet:

source s_snort_alert {
file("/var/log/snort/alert" tags("snort"));
};

rewrite r_snort_tag {
set("MSG_TAG", value("[SNORT] ") condition(tags("snort")));
};

source(s_snort_alert);
rewrite (r_snort_tag);

Thank you

On Thu, Nov 3, 2011 at 5:24 PM, Miguel Alvarez <miguellvrz9 at gmail.com> wrote:
> On Thu, Nov 3, 2011 at 4:03 PM, Balint Kovacs <balint.kovacs at balabit.com> wrote:
>>
>> On 11/03/2011 10:33 PM, Jakub Jankowski wrote:
>>> On 2011-11-03, Miguel Alvarez wrote:
>>>
>>>>> Well, something similar is possible with syslog-ng aswell: (beware,
>>>>> completely untested, there might be typos!)
>>> [...]
>>>>> source s_snort_alert {
>>>>>  file("/var/log/snort/alert");
>>>>>  tag("snort");
>>>>> };
>>> [...]
>>>>>  [1]:
>>>>> http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-3.2-guides/syslog-ng-ose-v3.2-guide-admin-en.html/bk01-toc.html
>>>>
>>>> Wow, thank you so much!  That actually looks pretty straight forward.
>>>>
>>>> I initially had syslog-ng 3.2.4 installed but it was complaining about
>>>> the "source plugin tag not found".  I thought this was perhaps due to
>>>> it not being 3.3 so I built and installed 3.3.1 but am still seeing
>>>> it.  Is there something I'm missing from my build or not loading in my
>>>> config?
>>>
>>> It's a typo in Gergely's example. It's "tags", not "tag" :) See the
>>> docs[2]
>>>
>>> [2]
>>> http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-3.2-guides/syslog-ng-ose-v3.2-guide-admin-en.html/reference_source_file.html
>> Also, tags() have to be within the file() declaration, so the above
>> should rather look like
>>
>> source s_snort_alert {
>>     file("/var/log/snort/alert" tags("snort"));
>> };
>
> Thank you, Balint.  I fixed the "tag" > "tags" as well as the
> "rewrite" sections.  Everything starts successfully and lsof shows the
> defined files are being opened but no logs are being forwarded.  I'm
> running tcpdump on the server side and see the three-way handshake and
> there's even a message in /var/log/messages saying that the connection
> was accepted:
>
> 22:39:03.520177 IP (tos 0x0, ttl  61, id 29965, offset 0, flags [DF],
> proto: TCP (6), length: 60) 192.168.1.2.46244 > 192.168.1.1.1200: S,
> cksum 0x9589 (correct), 2051237327:2051237327(0) win 5840 <mss
> 1460,sackOK,timestamp 1304367298 0,nop,wscale 10>
> 22:39:03.520201 IP (tos 0x0, ttl  64, id 0, offset 0, flags [DF],
> proto: TCP (6), length: 60) 192.168.1.1.1200 > 192.168.1.2.46244: S,
> cksum 0x07c4 (correct), 2951440826:2951440826(0) ack 2051237328 win
> 5792 <mss 1460,sackOK,timestamp 4114972411 1304367298,nop,wscale 7>
> 22:39:03.522092 IP (tos 0x0, ttl  61, id 29966, offset 0, flags [DF],
> proto: TCP (6), length: 52) 192.168.1.2.46244 > 192.168.1.1.1200: .,
> cksum 0x4d29 (correct), 1:1(0) ack 1 win 6 <nop,nop,timestamp
> 1304367299 4114972411>
>
> Nov  3 22:39:03 logserver syslog-ng[24827]: Syslog connection
> accepted; fd='44', client='AF_INET(192.168.1.2:46244)',
> local='AF_INET(192.168.1.1:1200)'
>
> Anyone have any suggestions on what I can try?
>
> And for the record, here's the updated config:
>
> ###
> # File sources
> # ------------
> #
> # These set up sources, and tag them appropriately. We'll use the tags
> # later in the rewrite rules.
> ###
>
> source s_snort_alert {
> file("/var/log/snort/alert" tags("snort"));
> };
>
> source s_bro_conn {
> file("/var/log/bro/conn.log" tags("bro-conn"));
> };
>
> source s_bro_http {
> file("/var/log/bro/http.log" tags("bro-http"));
> };
>
> ###
> # Templates
> # ---------
> #
> # Templates are used similarly as in rsyslog (except our templates are
> # awesome, and theirs isn't. Sadly, this example is too simple to show
> # the power of syslog-ng templates. Oh well..).
> #
> # Anyway, in this case, the template will be similar to a normal
> # BSD legacy syslog format, with ${MSG_TAG} inserted between the
> # MSGHEADER and the message itself. If MSG_TAG is unset, nothing will be
> # inserted, and we'll get a standard format.
> ###
> template t_tagged {
> template("${ISODATE} ${HOST} ${MSGHDR}${MSG_TAG}${MSG}");
> };
>
> ###
> # Destinations
> # ------------
> #
> # Ye olde TCP destination. You can replace tcp with upd, if so you
> # wish. It forwards everything that reaches the destination to the
> # specified host, on the given port, using the template we made above.
> ###
> destination d_remote_tagged {
>  tcp("192.168.1.1" port(1200) template(t_tagged));
> };
>
> ###
> # Rewrite
> # -------
> #
> # Rewrite rules! If we encounter a tag we care about, we set MSG_TAG
> # appropriately. That is all. If a message does not have the sought tag,
> # the rewrite does nothing.
> ###
> rewrite r_snort_tag {
> set("MSG_TAG", value("[SNORT] ") condition(tags("snort")));
> };
>
> rewrite r_bro_conn_tag {
> set("MSG_TAG", value("[BRO-CONN] ") condition(tags("bro-conn")));
> };
>
> rewrite r_bro_http_tag {
> set("MSG_TAG", value("[BRO-HTTP] ") condition(tags("bro-http")));
> };
>
> ###
> # Logpath
> # -------
> #
> # Logpaths define how sources, filters, rewrite rules and destinations
> # are connected.
> #
> # In this case, this logpath will read from all three file sources
> # defined above, pass them through all three rewrite rules (remember:
> # those only do the rewrite if the appropriate tag matches), and
> # finally, send it over to the remote host.
> #
> # We also set a "final" flag, which means that if a message was caught
> # by this rule (ie, it came from any of the three files), it will not be
> # processed further by any other logpath.
> ###
>
> log {
> source(s_snort_alert);
> source(s_bro_alarm);
> source(s_bro_conn);
> source(s_bro_http);
> source(s_bro_notice);
>
> rewrite (r_snort_tag);
> rewrite (r_bro_conn_tag);
> rewrite (r_bro_http_tag);
>
> destination (d_remote_tagged);
>
> flags(final);
> };
>