- syslog-ng - lists.balabit.hu

Help with pattern db correlation
by Martin Holste 18 Dec '10

18 Dec '10

I'm trying to build a correlation message for Ironport similar to the Postfix example that was referred to in the discussions. Here's what I have so far, but it's not quite working: <ruleset> <rules> <rule class="10" id="10" context-id="ironport-mid" context-timeout="10" context-scope="program"> <patterns> <pattern>Info: Start MID @NUMBER:mid:@ ICID @NUMBER:icid:@</pattern> </patterns> <values> <value name="icid">$icid</value> </values> <examples> <example> <test_message program="ironport_mail_logs">Info: Start MID 144753300 ICID 696117306</test_message> </example> </examples> </rule> <rule class="10" id="10" context-id="ironport-mid" context-timeout="10" context-scope="program"> <patterns> <pattern>Info: Message finished MID @NUMBER:mid:@ done</pattern> </patterns> <actions> <action> <message> <values> <value name="MESSAGE">IronPort message complete: icid: $icid, mid: $mid</value> </values> </message> </action> </actions> <examples> <example> <test_message program="ironport_mail_logs">Info: Message finished MID 144753300 done</test_message> </example> </examples> </rule> </rules> </ruleset> I'm getting the triggered action, but the icid is null while the mid is filled in. What am I missing?

3 4

load failures in afsocket and afsql
by Matthew Hall 17 Dec '10

17 Dec '10

I am getting the following on load: Error opening plugin module; module='afsocket', error='/home/y/lib64/syslog-ng/libafsocket.so: undefined symbol: libnet_build_ipv4' Error opening plugin module; module='afsql', error='/home/y/lib64/syslog-ng/libafsql.so: undefined symbol: dbi_result_free' Error opening plugin module; module='afsocket', error='/home/y/lib64/syslog-ng/libafsocket.so: undefined symbol: libnet_build_ipv4' The rpath looks OK: megahall@logproxy2:~$ readelf -a /home/y/lib64/syslog-ng/libafsocket.so | fgrep -i rpath 0x000000000000000f (RPATH) Library rpath: [/home/y/lib64] megahall@logproxy2:~$ readelf -a /home/y/lib64/syslog-ng/libafsql.so | fgrep -i rpath 0x000000000000000f (RPATH) Library rpath: [/home/y/lib64] megahall@logproxy2:~$ megahall@logproxy2:~$ ldd /home/y/lib64/syslog-ng/libafs* | fgrep -i '(dbi|net)' megahall@logproxy2:~$ The libraries are in a reasonable location: /home/y/lib64/libdbi.so.1.0.0 /home/y/lib64/libnet.so.1.5.0 /home/y/lib64/dbd/libdbdsqlite3.so /home/y/lib64/dbd/libdbdmysql.so /home/y/lib64/libnet.so.1 /home/y/lib64/libdbi.so.1 /home/y/lib64/libdbi.so Reading through the glib docs for glib modules, it seems like the .la files are maybe not containing the right library dependencies, or something like this. However adding the library directories using LD_LIBRARY_PATH as a temporary test does not help. Because this step fails, it's not possible to use tcp, udp, or any of the other important drivers you need to collect logs. I could really use some advice on this one! Matthew.

5 25

Help with db_parser()
by Giovanni Mancuso 17 Dec '10

17 Dec '10

Hi, I configure my Java application to log with log4j in syslog server, and i configure my syslog-ng to store data in mysql database, but this doen't work. My syslog-ng configuration is: @version: 3.0 options { chain_hostnames(no); stats_freq(43200); }; source src { unix-stream("/dev/log" max-connections(256)); internal(); udp(ip("127.0.0.1") port(514)); file("/proc/kmsg"); }; destination mcs { file("/var/log/mw-collaboration/mw-collaboration-loginfile.log"); }; destination mcs_sql { sql( type(mysql) host("localhost") username("syslogng") password("syslogng") database("mcslogin") table("mcslogin") columns("date varchar(32)","loginuser varchar(32) ","ipsource varchar(32)") values("${S_YEAR}-${S_MONTH}-${S_DAY} ${S_HOUR}:${S_MIN}:${S_SEC}","${LOGIN_USER}","${IP_SOURCE}") ); }; filter f_mcs { facility(local2); }; parser p_mcs { db_parser(file("/etc/syslog-ng/patterndb.d/mcs.xml")); }; log { source(src); filter(f_mcs); parser(p_mcs); destination(mcs_sql); destination(mcs); }; I created also a db_parser file that is: <patterndb version='1' pub_date='2010-12-14'> <program name='mcs'> <pattern>mcs</pattern> <rule id='mcs' class='system'> <pattern>###############Accesso dell'utente @STRING:LOGIN_USER@ da ip @IPv4:IP_SOURCE@</pattern> </rule> </program> </patterndb> With this configuration, i have that in "mcs" destination it writes all information: Dec 16 11:55:44 localhost mcs[123] ###############Accesso dell'utente xxxx(a)xxx.mailware.it da ip 111.222.333.444 Dec 16 12:53:23 localhost mcs[123] ###############Accesso dell'utente xxxx.1(a)xxx.mailware.it da ip 111.222.333.444 Dec 16 14:07:40 localhost mcs[123] ###############Accesso dell'utente xxxx.1(a)xxx.mailware.it da ip 111.222.333.444 but in "mcs_sql" destination, it writes only date: mysql> desc mcslogin; +-----------+-------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-----------+-------------+------+-----+---------+-------+ | date | varchar(32) | YES | MUL | NULL | | | loginuser | varchar(32) | YES | | NULL | | | ipsource | varchar(32) | YES | | NULL | | +-----------+-------------+------+-----+---------+-------+ 3 rows in set (0.00 sec) mysql> select * from mcslogin limit 10 -> ; +---------------------+-----------+----------+ | date | loginuser | ipsource | +---------------------+-----------+----------+ | 2010-12-15 11:02:16 | | | | 2010-12-15 11:11:09 | | | | 2010-12-15 17:53:01 | | | | 2010-12-15 18:11:55 | | | | 2010-12-15 18:12:54 | | | | 2010-12-15 18:35:07 | | | | 2010-12-16 11:55:36 | | | | 2010-12-16 11:55:44 | | | | 2010-12-16 11:55:44 | | | | 2010-12-16 12:53:23 | | | +---------------------+-----------+----------+ 10 rows in set (0.00 sec) Can you help me? Thanks

3 6

memory exhausted in /opt/syslog-ng/etc/syslog-ng.conf at line 5347
by Mukarram Syed 17 Dec '10

17 Dec '10

Hi , I am getting the following error when starting syslog-ng.conf file which has more than 5000 lines in it and getting the following error: # service syslog-ng start memory exhausted in /opt/syslog-ng/etc/syslog-ng.conf at line 5347. syslog-ng documentation: http://www.balabit.com/support/documentation/?product=syslog-ng mailing list: https://lists.balabit.hu/mailman/listinfo/syslog-ng [FAILED] When I truncate the file to < 5000 lines it starts up fine. This is running version syslog-ng-3.0.8-1.rhel5.x86_64 Mailing list does not have any clues to it. Anything in the syslog-ng.conf file that needs to be done to accept more lines? Any ideas would be appreciated. Thanks # mukarram Mukarram Syed

6 23

Patch for launch path bug in syslog-ng.h
by Matthew Hall 17 Dec '10

17 Dec '10

Hello, When compiling syslog-ng 3.2.1 into my environment I discovered a bug in how this configure option is applied: --libexecdir=/home/y/libexec64 When this option is entered, this is generated in lib/syslog-ng.h: #if ENABLE_ENV_WRAPPER #define PATH_SYSLOGNG PATH_PREFIX "/libexec/syslog-ng" #endif As you can see this is a hard coded libexec path which does not match what my environment expects. Unfortunately it appears that the libexec variable never makes it into a C macro so I'm not totally clear on how to make a permanent patch for this one. I will work around it with something like this temporary patch for now. --- /home/megahall/syslog-ng.h 2010-12-13 15:54:56.000000000 -0800 +++ syslog-ng-3.2.1/lib/syslog-ng.h 2010-12-13 16:06:22.000000000 -0800 @@ -44,7 +44,7 @@ #define PATH_PIDFILE PATH_PIDFILEDIR "/syslog-ng.pid" #define PATH_CONTROL_SOCKET PATH_PIDFILEDIR "/syslog-ng.ctl" #if ENABLE_ENV_WRAPPER -#define PATH_SYSLOGNG PATH_PREFIX "/libexec/syslog-ng" +#define PATH_SYSLOGNG PATH_PREFIX "/libexec64/syslog-ng" #endif #define PATH_PERSIST_CONFIG PATH_LOCALSTATEDIR "/syslog-ng.persist" Another issue. Because of this bug syslog-ng fails to launch with return code 127. But it prints nothing to the console explaining why it failed: execve("/home/y/libexec/syslog-ng", ["/home/y/sbin64/syslog-ng"], [/* 21 vars */]) = -1 ENOENT (No such file or directory) exit_group(127) = ? I think some message should be added in syslog-ng/wrapper.c. This is an uncompiled example but should be very close. --- /home/megahall/wrapper.c 2010-12-13 16:15:29.000000000 -0800 +++ syslog-ng-3.2.1/syslog-ng/wrapper.c 2010-12-13 16:14:21.000000000 -0800 @@ -45,6 +45,7 @@ putenv(ldlibpath); } #endif + fprintf(stderr, "syslog-ng failed to launch because of XYZ\n"); execv(PATH_SYSLOGNG, argv); return 127; } Thanks for maintaining syslog-ng. I hope this helps keep the product great! Regards, Matthew Hall.

2 2

Convert syslog to traps
by Jay 14 Dec '10

14 Dec '10

Have a requirement to convert all incoming syslogs to SNMP traps and send it to another host. One option I could think of is to use program () destination. When I tried this option, I find that syslog-ng is continuously sending newline characters to the specified program. i.e. even when no syslog is received, syslog-ng seems to be pumping newline chars to the specified program. Also I read the warning message in admin guide that, it will open up the door to DOS attack. Could someone let me know the best way to achieve this, please ? Thanks in advance for the help. John

5 8

Microsoft SQL connect issue
by Shawn Cannon 14 Dec '10

14 Dec '10

When a Microsoft SQL connection is attempted, I get the following errors. I am trying to connect to a names instance of MS SQL over TCP 1785. Any ideas? Thanks.... Dec 13 10:33:23 syslog01 syslog-ng[27963]: Error running SQL query; type='freetds', host='sql.hidden.local\instance', port='1785', user='user', database='syslog', error='', query='CREATE TABLE syslogng (datetime varchar(16), host varchar(32), program varchar(32), pid varchar(8), message varchar(4096))' Dec 13 10:33:23 syslog01 syslog-ng[27963]: Error creating table, giving up; table='syslogng' Dec 13 10:33:23 syslog01 syslog-ng[27963]: Error checking table, disconnecting from database, trying again shortly; time_reopen='60' Shawn Cannon IT Professional http://www.shawncannon.com Contact Me <http://www.linkedin.com/pub/shawn-cannon/5/84/81a> LinkedIn<http://www.linkedin.com/pub/shawn-cannon/5/84/81a> <http://www.facebook.com/rolltidega72> Facebook<http://www.facebook.com/rolltidega72> <http://twitter.com/rolltidega> Twitter <http://twitter.com/rolltidega> My blog: My thoughts on VMworld… a week later…<http://www.shawncannon.com/?p=19>

3 13

Help configuring a basic setup
by Shawn Cannon 10 Dec '10

10 Dec '10

OK I want to setup syslog-ng to listen for syslog traffic on the standard UDP port and have the messages received logged to a Microsoft SQL database. I am having trouble finding documentation I can understand. I am a very GUI driven admin so all the command line stuff is messing with me! If anyone can assist I would appreciate it. Thanks.... Shawn Cannon IT Professional http://www.shawncannon.com Contact Me <http://www.linkedin.com/pub/shawn-cannon/5/84/81a> LinkedIn<http://www.linkedin.com/pub/shawn-cannon/5/84/81a> <http://www.facebook.com/rolltidega72> Facebook<http://www.facebook.com/rolltidega72> <http://twitter.com/rolltidega> Twitter <http://twitter.com/rolltidega> My blog: My thoughts on VMworld… a week later…<http://www.shawncannon.com/?p=19>

1 0

Re: [syslog-ng] Creating a "log" block (OSE 3.2.1)
by Eric Berggren 10 Dec '10

10 Dec '10

> thanks for noticing this error. I've corrected it in the source repo of the > adminguide, and will update the docs on the website when someone tells me if > using {} within blocks is permitted (but now buggy) or forbidden. > changing the type of the block to root should workaround the problem. I see, it's really about using { }'s within the block definition. Making it a "root" block (as a completely separately log {} stanza) is probably cleaner in this particular case, but the same restriction applies at the moment -- the example from the OSE 3.2 documentation [5.3.2]: block root mylogs() { source s_file { file("/var/log/mylogs.log" follow_freq(1)); }; destination d_local { file("/var/log/messages"); }; log { source(s_file); destination(d_local); }; }; doesn't parse either (same error on the final "};") The documentation (html/PDF) could use some minor updates in this area for clarity: - block definitions also require ()'s after the name (including non-parameterized); e.g. block root foo() { ... } - perhaps add an additional example for using parameterized blocks; e.g. block destination foo ( logname("some.log") ) { file( "/var/log/`logname`" ... ); }; destination bar { foo( logname("other.log") ) }; destination baz { foo() }; regards, -eric ps: I have yet to actually *test* any of this new functionality myself yet (later today :)

3 4

Re: [syslog-ng] Creating a "log" block (OSE 3.2.1)
by Eric Berggren 10 Dec '10

10 Dec '10

Admittedly, I ended up just using a separate log {} statement (obviating the need for the block statement in this particular instance, but using blocks successfully elsewhere -- very nice). I tested the earlier lexer patch and found it works (as I imagined you did), and sure I'll find more creative uses . Allowing the extra ";" when referencing blocks indeed makes it look more consistent . thanks again -eric > I've just committed these 3 patches that implement block support for > block statements. I've also resolved the '{' issue and also made it > possible to use several ';' characters in place of a single one.

1 0