[syslog-ng] Properly Setting Facility and Severity into destination

Tue Aug 1 15:29:55 UTC 2023

Hello,

I must admit I'm not sure that I understood everything you wrote. So, without knowing what you mean by a regular application log (which is kind of ambigous as even a single event could get splitted into hundreds of lines especially when it comes to backtraces) assuming one unstructured/ unformatted thus random line representing an event the worst-case scenario isusing the no-parse flag to read the entire line into the $MESSAGE macro and build an rfc5424-complant message using a template like this:

"<81>1 $ISODATE $HOST fakeapplication fakeprocessid fakemsgid - $MESSAGE\n"

obviously nobody likes when the timestamp doesn't match the date when the actual event happened but when the log processor ingested it (logstash @timestamp and syslog output plugin I'm looking at you!)

So if the input contain somewhat syslog-looking messages (contains a timestamp, application name, etc.) then you can experiment with syslog-parser(). The last resort would be writing a custom parser.

Without seeing an actual log message and a decoded packet reaching the syslog server how it was transformed by the client I can't help.

Note that syslog-ng internally defines the facility names so "security" is facility 13 so the expected priority should be between 104 and 111, "authpriv" has facility code 10. This is documented in the OSE admin guide, not just in lib/syslog-names.c

Regards,
Sandor

On 2023. 08. 01. 1:09, Greg Christopher wrote:
Double checking; is the list dead? I did not receive a bounce.

On Saturday, July 29, 2023 at 11:06:28 PM GMT+9, Greg Christopher <gregory_christopher at yahoo.com><mailto:gregory_christopher at yahoo.com> wrote:

Hi All,
   Did my best (about a week effort) to find this information online and apologize if it's obvious.

   I am trying to use syslog-ng as a client to send messages to a syslog compliant server with the proper severity and facility set.  I am starting with plain-text log files.

   The syslog rfc<https://www.rfc-editor.org/rfc/rfc5424> as well as the syslog-ng documentation<https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454445555731-4879cd1edfcd8661&q=1&e=290c4c0e-1f5b-482c-8902-b0c3bc63b102&u=https%3A%2F%2Fwww.syslog-ng.com%2Ftechnical-documents%2Fdoc%2Fsyslog-ng-open-source-edition%2F3.18%2Fadministration-guide%2F8> are pretty clear about the syslog message format itself.

   There are even syslog-ng functions to substitute severity and facility<https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454445555731-39073d966e982428&q=1&e=290c4c0e-1f5b-482c-8902-b0c3bc63b102&u=https%3A%2F%2Fwww.syslog-ng.com%2Ftechnical-documents%2Fdoc%2Fsyslog-ng-open-source-edition%2F3.36%2Fadministration-guide%2F77%23TOPIC-1768796> using a function called "rewrite". But this doesn't seem to work if you are starting with a regular application log. In other words, there is nothing to "rewrite" since this header was never there to begin with.
   Although I initially configured my application log with flags(no-parse) , I attempted to remove it so that my rewrite attempt would at least have a facility and severity to "rewrite", but this also seems to have failed.

   Next I attempted prepending "<81>" (8*10 + 1) to the application messages to get the right priority field but this failed to change what the syslog-ng server on the other end received, as I have modified its output to include severity and facility:
---
destination d_local {
        file("/var/log/syslog-ng/$HOST" template("Fac: $FACILITY Pri: $PRIORITY $ISODATE $HOST $(format-welf --scope all-nv-pairs)\n") frac-digits(3));
};
---
   It does seem to be working properly but the above two approaches to increasing the severity and noting this is a security facility message failed.

So it seems this use case where we create syslog entries from scratch sort of throws a curve ball at syslog-ng clients, at least from what I can figure.

Any help on how I can set these manually into the destination would be greatly appreciated.

Here is my syslog-ng.conf at the moment; note I pull log file name out of environment variable:
---
@version: 4.1

# Configure the source to read from the messages log file
# no-parse means whole line becomes "message" portion.
# Template should indicate security alert to syslog daemon
source s_APP_LOG {
    file(`APP_LOG`);
#    file(`APP_LOG` flags(no-parse));

};

rewrite set_pri_fields {
        set-severity("alert");
        set-facility("security");
};

destination d_destination {
    syslog(`syslogServer`);
};

# Configure the log statement to route messages from the application log file to
# the syslog server specified on the command line

log {
    source(s_APP_LOG);
    rewrite(set_pri_fields);
    destination(d_destination);
};

---
Greg

______________________________________________________________________________
Member info: https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454445555731-72681b0c6466e37f&q=1&e=290c4c0e-1f5b-482c-8902-b0c3bc63b102&u=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng
Documentation: https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454445555731-495b5472c8de2e06&q=1&e=290c4c0e-1f5b-482c-8902-b0c3bc63b102&u=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng
FAQ: https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454445555731-c93a94005a04ead7&q=1&e=290c4c0e-1f5b-482c-8902-b0c3bc63b102&u=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20230801/de6db0b7/attachment-0001.htm>