hi, On Tue, 2010-02-02 at 08:36 -0600, Jeremy M. Guthrie wrote:

Sorry, I didn't see an answer. Just wanted to re-post to reconfirm:

I have an issue where what is being written to file is different than what I have being written to disk. I don't know if the macro is broken, my understanding is wrong, or I am just using the wrong macro altogether.

I guess you meant here that file/SQL is different, right?

My understanding is roughly as follows: PROGRAM The name of the program sending the message. PID The PID of the program sending the message. MSG or MESSAGE Message contents including the program name and pid. MSGONLY Message contents without the program name.

this is true for 2.0 and 2.1, syslog-ng 3.0 changed this behaviour and although it can operate in 2.1 compatibility mode, it all depends on the syslog-ng.conf file, whether it contains '@version: 3.0' at the front. If there's no @version directive, or it contains '2.1', the things you wrote above is true. If you are running syslog-ng in 3.0 mode (which performs better), then $MSG: doesn't contain the program name and pid any more, basically it is the same as $MSGONLY $MSGHDR: contains the combined program/pid value and of course you do have the separate macros ($PROGRAM, $PID)

1. The first thing I ran into: Example messages on file: Jan 25 08:34:15 XXXXXXXXXX sshd[7209]: service name is :: sshd

Same example message in SQL: Jan 25 08:34:15 XXXXXXXXXX service name is :: sshd

template("INSERT INTO tsyslog (host, facility, severity, date, time, message) VALUES ( '$HOST','$FACILITY_NUM','$LEVEL_NUM','$YEAR-$MONTH-$DAY','$HOUR:$MIN: $SEC', '$MSG');\n" )

did you specify a template for your file? if you didn't then this is correct 3.0 behaviour.

2. Then what I tried to get it closer.... Example messages on file: Jan 25 08:40:18 XXXXXXXXXX sshd[7209]: service name is :: sshd

Same example message in SQL: Jan 25 08:40:18 XXXXXXXXXX sshd: service name is :: sshd

template("INSERT INTO tsyslog (host, facility, severity, date, time, message) VALUES ( '$HOST','$FACILITY_NUM','$LEVEL_NUM','$YEAR-$MONTH-$DAY','$HOUR:$MIN: $SEC', '$PROGRAM: $MSG');\n" )

again. this is correct.

3. Then what I tried to get even closer: Example messages on file: Jan 25 08:53:15 XXXXXXXXXX sshd[7209]: service name is :: sshd

Same example message in SQL: Jan 25 08:53:15 XXXXXXXXXX sshd[7209]: service name is :: sshd

template("INSERT INTO tsyslog (host, facility, severity, date, time, message) VALUES ( '$HOST','$FACILITY_NUM','$LEVEL_NUM','$YEAR-$MONTH-$DAY','$HOUR:$MIN: $SEC', '$PROGRAM[$PID]: $MSG');\n" )

correct again.

However, this broke messages coming from Cisco devices... Jan 25 08:55:10 FIREWALL %FWSM-3-710003[]: tcp access denied by ACL from 188.23.180.240/3826 to outside:172.26.54.65/445

The message above should be: Jan 25 08:55:10 FIREWALL %FWSM-3-710003: tcp access denied by ACL from 188.23.180.240/3826 to outside:172.26.54.65/445

Notice the above '[]' after the FWSM code? I am running 3.0.5 syslog-ng.

Ideally I'd get both: Jan 25 08:55:10 FIREWALL %FWSM-3-710003: tcp access denied by ACL from 188.23.180.240/3826 to outside:172.26.54.65/445

AND

Jan 25 08:53:15 XXXXXXXXXX sshd[7209]: service name is :: sshd

Is the MSG/MESSAGE macro broken? What am I missing?

you could perhaps use the $MSGHDR macro, which combines the program/pid fields and omits the '[]' part if it is not needed. Also you could possibly need the 'store-legacy-msghdr' macro on your input to make it absolutely sure that the log message remains intact. (without this flag, syslog-ng is trying to parse/reconstruct the program[pid] part, which may be slightly different than the original input). -- Bazsi