- syslog-ng - lists.balabit.hu

RE: [syslog-ng] Re: question regarding program name logs
by Justin Randall 27 Sep '06

27 Sep '06

Is it maybe possible that this is a new issue not related to the initial one outlined in this thread? It occurred to me that the messages that are "disappearing" are clearly not behaving in the same way as in 2.0rc1, where they simply went to the wrong place because of an angle bracket tag with digits inside. In this case it simply seems like the first message in a burst of UDP messages is being completely lost. Since I've downgraded back to 2.0rc2, the problem no longer exists. Anything else I can do to help? Regards, Justin. -----Original Message----- From: Justin Randall Sent: Wednesday, September 27, 2006 10:10 AM To: Syslog-ng users' and developers' mailing list Subject: RE: [syslog-ng] Re: question regarding program name logs Sorry I pasted the wrong "log" statement, here it is: log { source(src_udp); source(src_tcp); filter(fltr_PROGRAM); destination(dst_ PROGRAM); flags(final, flow-control); }; -----Original Message----- From: Justin Randall Sent: Wednesday, September 27, 2006 10:07 AM To: Syslog-ng users' and developers' mailing list Subject: RE: [syslog-ng] Re: question regarding program name logs Hi and thanks for the response. This is actually relating to UDP logging. I have devices sending Syslog messages via UDP to a central Syslog-NG server. Usually the devices log messages to Syslog-NG via UDP in bursts (about 20-25 message within a couple seconds). The source, filter, destination, log statements for the handling are as follows: source src_udp { udp( ip(IPADDRESS) port(PORT) ); }; filter fltr_PROGRAM { program("PROGRAM") or match("PROGRAM"); }; destination dst_PROGRAM { file( "/var/log/PROGRAM/messages" owner(root) group(tomcat) perm(0640) ); }; log { source(src_udp); source(src_tcp); destination(dst_remote_system); flags(final, flow-control); }; What ends up happening when a host falls into this flow, the first log message is missed. The format of the bursts of messages are exactly the same. This did not happen in 2.0rc2, I have downgraded and verified this. Let me know if there's any other info I can provide that would help out. Regards, Justin. -----Original Message----- From: G.W. Haywood [mailto:ged@jubileegroup.co.uk] Sent: Wednesday, September 27, 2006 7:42 AM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng] Re: question regarding program name logs Hi there, On Wed, 27 Sep 2006 Justin Randall wrote: > Can anyone confirm if the patch that was to fix this issue was included > in 2.0rc3? Yes, it was. Note that it only affects logging via UDP. -- 73, Ged. _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

1 0

RE: [syslog-ng] Re: question regarding program name logs
by Justin Randall 27 Sep '06

27 Sep '06

Hi and thanks for the response. This is actually relating to UDP logging. I have devices sending Syslog messages via UDP to a central Syslog-NG server. Usually the devices log messages to Syslog-NG via UDP in bursts (about 20-25 message within a couple seconds). The source, filter, destination, log statements for the handling are as follows: source src_udp { udp( ip(IPADDRESS) port(PORT) ); }; filter fltr_PROGRAM { program("PROGRAM") or match("PROGRAM"); }; destination dst_PROGRAM { file( "/var/log/PROGRAM/messages" owner(root) group(tomcat) perm(0640) ); }; log { source(src_udp); source(src_tcp); destination(dst_remote_system); flags(final, flow-control); }; What ends up happening when a host falls into this flow, the first log message is missed. The format of the bursts of messages are exactly the same. This did not happen in 2.0rc2, I have downgraded and verified this. Let me know if there's any other info I can provide that would help out. Regards, Justin. -----Original Message----- From: G.W. Haywood [mailto:ged@jubileegroup.co.uk] Sent: Wednesday, September 27, 2006 7:42 AM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng] Re: question regarding program name logs Hi there, On Wed, 27 Sep 2006 Justin Randall wrote: > Can anyone confirm if the patch that was to fix this issue was included > in 2.0rc3? Yes, it was. Note that it only affects logging via UDP. -- 73, Ged. _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

1 0

file issues
by Brian Loe 27 Sep '06

27 Sep '06

I'm running into all kinds of problems running syslog-ng. It won't create the files in subdirectories like I want, it won't feed the named pipes I want... The following is my configuration (and I'm running sec against the pipes - since I need a reader before I start the writer on AIX): # cat /etc/syslog-ng/syslog-ng.conf options { sync (0); chain_hostnames (no); use_dns (no); use_fqdn (no); create_dirs (yes); keep_hostname (yes); dir_owner(syslog); dir_group(syslog); dir_perm(0660); owner(syslog); group(syslog); perm(0660); }; source s_cron { unix-dgram("/dev/cron"); internal(); }; source s_snmp { unix-dgram("/dev/snmp"); internal(); }; source s_sys { unix-dgram("/dev/log"); internal(); }; source s_udp { udp(ip("0.0.0.0") port(514)); }; source s_udp_s { udp(ip("0.0.0.0") port(601)); }; source s_tcp { tcp(ip("0.0.0.0") port(601)); }; source s_tcp1 { tcp(ip("0.0.0.0") port(1026)); }; source s_tcp2 { tcp(ip("0.0.0.0") port(5140)); }; destination d_cons { file("/data/logs/console" perm(0660)); }; destination d_mesg { file("/data/logs/syslog.log" perm(0660)); }; destination d_mail { file("/data/logs/mail.log" perm(0660)); }; destination d_mlrt { usertty("root"); }; destination d_mlal { usertty("*"); }; destination allmessages { file("/data/logs/$YEAR$MONTH$DAY.allmessages.log" owner(syslog) group(syslog) perm(0660)); }; destination netmessages { file("/data/logs/$YEAR$MONTH$DAY.netmessages.log" owner(syslog) group(syslog) perm(0660)); }; destination d_vpn { file("/data/logs/vpn/$YEAR$MONTH$DAY.vpn.log" owner(syslog) group(syslog) perm(0660)); }; destination d_switch { file("/data/logs/switch/$YEAR$MONTH$DAY.switch.log" owner(syslog) group(syslog) perm(0660)); }; destination d_router { file("/data/logs/router/$YEAR$MONTH$DAY.router.log" owner(syslog) group(syslog) perm(0660)); }; destination d_pixen { file("/data/logs/pixen/$YEAR$MONTH$DAY.pixen.log" owner(syslog) group(syslog) perm(0660)); }; destination d_ids { file("/data/logs/ids/$YEAR$MONTH$DAY.ids.log" owner(syslog) group(syslog) perm(0660)); }; destination vpn_pipe { pipe("/data/pipes/vpn_pipe"); }; # destination net_pipe { pipe("/data/pipes/net_pipe"); }; destination fire_pipe { pipe("/data/pipes/fire_pipe"); }; filter f_filter1 { facility(mail) and level(debug); }; filter f_filter2 { (facility(mail) and level(debug)) or level(info); }; filter f_filter3 { level(alert); }; filter f_filter4 { level(emerg); }; filter f_fl2r16 { netmask(1.1.253.5/255.255.255.255); }; filter f_fl2r17 { netmask(1.1.253.6/255.255.255.255); }; filter f_fl2f12 { netmask(1.1.253.2/255.255.255.255); }; filter f_fl2f1 { netmask(1.1.253.2/255.255.255.255); }; filter f_fl2f2 { netmask(1.1.62.10/255.255.255.255); }; filter f_fl2f11 { netmask(1.1.193.10/255.255.255.255); }; filter f_fl1f1 { netmask(1.1.253.212/255.255.255.255); }; log { source(s_sys); filter(f_filter1); destination(d_mail); }; log { source(s_sys); filter(f_filter2); destination(d_mesg); }; log { source(s_sys); filter(f_filter3); destination(d_cons); destination(d_mlrt); }; log { source(s_sys); filter(f_filter4); destination(d_mlal); }; # log { source(s_sys); source(s_cron); source(s_snmp); destination (allmessages); }; log { source(s_sys); source(s_cron); destination (allmessages); }; log { source(s_udp); source (s_udp_s); source (s_tcp); source (s_tcp1); source (s_tcp2); destination(netmessages); }; # log { source(s_udp); source (s_udp_s); source (s_tcp); source (s_tcp1); source (s_tcp2); destination(net_pipe); }; log { source(s_udp); filter(f_fl2r16); filter(f_fl2r17); filter(f_fl2f12); destination(vpn_pipe); }; log { source(s_udp); filter(f_fl2r16); filter(f_fl2r17); filter(f_fl2f12); destination(d_vpn); }; log { source(s_udp); filter(f_fl2f1); filter(f_fl2f2); filter(f_fl2f11); filter(f_fl2f12); filter(f_fl1f1); destination(fire_pipe); }; log { source(s_udp); filter(f_fl2f1); filter(f_fl2f2); filter(f_fl2f11); filter(f_fl2f12); filter(f_fl1f1); destination(d_pixen); }; The command I'm running (and I'm TRYING to run it from inittab): syslog-ng:23456789:respawn:/usr/local/sbin/syslog-ng -f /etc/syslog-ng/syslog-ng.conf -u syslog -g <me> -p /etc/syslog-ng.pid

2 10

RE: [syslog-ng] Re: question regarding program name logs
by Justin Randall 26 Sep '06

26 Sep '06

Can anyone confirm if the patch that was to fix this issue was included in 2.0rc3? Justin -----Original Message----- From: Justin Randall Sent: Monday, September 25, 2006 11:46 AM To: Syslog-ng users' and developers' mailing list Subject: RE: [syslog-ng] Re: question regarding program name logs After upgrading and monitoring, the logs which would previously get logged incorrectly due to the angle bracket prefixing are now not being logged at all. These logs don't come in with any peculiarity about them. The only thing if interest to note is that it is always the first log message in a group of log messages (i.e. host sends 5 Syslog messages, the first one is missed). What kind of logs, information would be helpful to diagnose? I'm currently running on 2.0rc3. Regards, Justin. -----Original Message----- From: G.W. Haywood [mailto:ged@jubileegroup.co.uk] Sent: Tuesday, September 12, 2006 6:27 AM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng] Re: question regarding program name logs Hi there, On Tue, 12 Sep 2006 Justin Randall wrote: > I am currently using Syslog-NG version 2.0rc1. > > I've noticed that sometimes when Syslog-NG is logging messages, it > prefixes something onto the program name which disrupts filters > while matching against a program name. For example, in some logs > for a program named "foo" I'll see logs in the form of "<15>foo" > which fail to be matched against a filter of 'program("foo") '. > This seems to only be noticed on logs which originate from remote > hosts. > > Could anyone provide an explanation for what is going on with this? I reported this problem a couple of weeks ago, and Balazs kindly provided a patch which fixed it. I imagine the patch is in .rc2, try upgrading. -- 73, Ged. _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

1 0

template-escape option and tabs problem
by Manuel Mora 22 Sep '06

22 Sep '06

Hi, I'm using syslog-ng as a central logging server, we are redirectioning some machine's logs to a FIFO pipe that is connected to a MySQL DB via a bash script (the typical syslog-ng , MySQL and php-syslog-ng scenario). destination d_mysql { pipe("/tmp/mysql.pipe" template("INSERT INTO logs (host, facility, priority, level, tag, datetime, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); }; Our Windows machines are forwarding logs to the central logging server using SNARE and SNARE uses horizontal tabs as field separator. Early we started to notice that the messages had a strange format in our DB so we redirectioned logs to a file using the same template to check for problems. In the field corresponding to '$MSG' we obtained the next message (with \011\ instead of tabs) : 'EMGDCW502.esp.e-corpnet.org\011MSWinEventLog\0111\011Security\01111688642\011Wed Sep 06 11:20:06 2006\011540\011Security\011ANONYMOUS LOGON\011Well Known Group\011Success Audit\011EMGDCW502\011Logon/Logoff\011\011Successful Network Logon: User Name: Domain: Logon ID: (0x1,0xFAC17236) Logon Type: 3 Logon Process: NtLmSsp Authentication Package: NTLM Workstation Name: EMCANW501 Logon GUID: - Caller User Name: - Caller Domain: - Caller Logon ID: - Caller Process ID: - Transited Services: - Source Network Address: 10.210.32.230 Source Port: 0 \01111688641' If we use template-escape(no) the message was received correctly so there is a problem with the parsing when template-escape is set to yes, it affects tab characters and it should affect only to ' and " characters. Are there any solutions for this? Best Regards. Manuel Mora

2 3

syslog-ng-2.0rc2 terminates if network connection is down
by Bruce Penrod 22 Sep '06

22 Sep '06

I don't think this is the way it should work... If I have a destination configured over a network path, say udp, and the ethernet device is misconfigured, or deliberately not configured, syslog-ng terminates immediate after start-up with these messages to the console: syslog-ng Connection failed; error='Network is unreachable (101)', reconnect='60' This happens even though other destinations are configured as well, like the local log files. Unfortunately, it doesn't manage to log this termination message in these local logs before it stops. I would think that the inability to reach any of the destinations, local or remote, would not cause the logger to die, if there is still some place it can log. -- Bruce Penrod

1 0

Re: NewBe syslog-ng and fedora core 5 64+bit
by G.W. Haywood 21 Sep '06

21 Sep '06

Hi there, On Thu, 21 Sep 2006 Gregory Machin wrote: > unresolved reference: inputs This tells you the problem. > this is my config file > ... > #source inputs { internal(); ... This 'source' line is commented out, so it doesn't exist, so you can't use the source 'inputs'. That will be a problem when you try to use it. Also the internal() source is required so you MUST include it. If you don't include it, that will be a problem whatever you do. > ... > log { source(inputs); > ... > log { source(inputs); > ... > log { source(inputs); > ... Here are three places where you try to use it. Why was it necessary to ask? If something isn't clear about any of that I'm sure people here would like to know, so that the clarity might be improved. > How should this be defined for fedora core ? > source stdlog { unix-dgram("/dev/log"); }; I don't use Fedora. I would think that's fine, but others here will know for sure. You might want to try log { source(stdlog); ... somewhere instead of one of your log { source(inputs); so that you actually use your 'stdlog' source. FWIW if you're having trouble understanding the syntax of syslog-ng's configuration file then I think you're going to run into difficulty if you try to log to MySQL as a first step. Why not try logging to files to start with, things will probably be a lot clearer for you. You can add database logging later, when you have things under control. -- 73, Ged.

2 1

NewBe syslog-ng and fedora core 5 64+bit
by Gregory Machin 21 Sep '06

21 Sep '06

Hi This is the first time I'm using syslog-ng and I'm trying to log all data to a mysql database ... when I start the service I get the following errors .. [root@greg syslog-ng]# service syslog-ng restart Stopping syslog-ng: [FAILED] Starting syslog-ng: Warning: No source refers to internal messages, they'll go to /dev/null unresolved reference: inputs Error initializing configuration, exiting. [FAILED] [root@greg syslog-ng]# this is my config file # syslog-ng central log host configuration file # # # ##################################################################### # General options options { check_hostname(yes); keep_hostname(yes); chain_hostnames(no); }; # Other important general options # use_fqdn(yes); # sync(0); # keep_hostname(yes); # chain_hostnames(no); # create_dirs(yes); ##################################################################### # Define syslog input stream # This is specific to Solaris 9 #source inputs { internal(); # sun-streams("/dev/log" door("/var/run/syslog_door")); # tcp(max_connections(1000)); }; # For newer versions of Linux (Debian GNU/Linux woody, # and RedHat Linux post 7.0) # source stdlog { unix-dgram("/dev/log"); }; <<<<<<<<<<<<<<<< How should this be defined for fedora core ? ##################################################################### # Define filters # Level Filters filter f_emerg { level (emerg); }; filter f_alert { level (alert .. emerg); }; filter f_crit { level (crit .. emerg); }; filter f_err { level (err .. emerg); }; filter f_warning { level (warning .. emerg); }; filter f_notice { level (notice .. emerg); }; filter f_info { level (info .. emerg); }; filter f_debug { level (debug .. emerg); }; # Facility Filters filter f_kern { facility (kern); }; filter f_user { facility (user); }; filter f_mail { facility (mail); }; filter f_daemon { facility (daemon); }; filter f_auth { facility (auth); }; filter f_syslog { facility (syslog); }; filter f_lpr { facility (lpr); }; filter f_news { facility (news); }; filter f_uucp { facility (uucp); }; filter f_cron { facility (cron); }; filter f_local0 { facility (local0); }; filter f_local1 { facility (local1); }; filter f_local2 { facility (local2); }; filter f_local3 { facility (local3); }; filter f_local4 { facility (local4); }; filter f_local5 { facility (local5); }; filter f_local6 { facility (local6); }; filter f_local7 { facility (local7); }; # syslog-ng stats filter filter f_n_stats { not match("STATS: dropped 0"); }; # syslog-ng "connect from filter filter f_n_connected { not match("AF_INET client connected from"); }; ##################################################################### # Define where the logs can be stored # Log log files destination logpile { file("/var/syslog-ng/$HOST/$YEAR/$MONTH/$FACILITY.$YEAR$MONTH$DAY" owner(root) group(root) perm(0600) create_dirs(yes) dir_perm(0700)); }; # Console log destination console { file("/dev/sysmsg"); }; # Linux Console # destination console { file("/dev/console"); }; # MySQL Database destination d_mysql { pipe("/var/run/mysql.pipe" template("INSERT INTO logs (host, facility, priority, level, tag, date, time, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL','$TAG', '$YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); }; ##################################################################### # Define logging # Local log log { source(inputs); filter(f_n_stats); filter(f_notice); filter(f_n_connected); destination(logpile); }; # Console log { source(inputs); filter(f_n_stats); filter(f_err); filter(f_n_connected); destination(console); }; # MySQL Database log { source(inputs); filter(f_n_stats); filter(f_notice); filter(f_n_connected); destination(d_mysql); }; -- Gregory Machin gregory.machin(a)gmail.com www.linuxpro.co.za

3 4

RE : PHP Syslog NG error's
by Manuel Mora 19 Sep '06

19 Sep '06

Hi Stuart, I have installed php-syslog-ng on a Suse 9 with no problems but in a Suse 10 (over a VMWARE) I had the 1st problem you have, I solved it disabling the NAME CACHE. Go to <document root>/include/config.php and seek for the line define('USE_CACHE', TRUE) and set it to define('USE_CACHE', FALSE). The second problem is strange, please, set logging in mysql, when mysql daemon is started you set the --log flag, then you'll have logs stored in the same directory where your data is. This log file contains the queries the clients do against the BD, you can check it for problems. Sorry for my english, if you don't understand something I can rewrite it. Regards. On 9/19/06, Stuart Boyle <stuart.boyle(a)cybertrust.com> wrote: > > > > Im having a couple of issue's with phpsyslog-ng, I cant seen any threads or > posts that help with the problem's im having so I need to post, if im > covering old ground, I apologize. > > Im not well versed with MySQL or PHP, so if you could help please explain in > "novice language". > > 1. When I open the browser and login, I then select "search" from the menu > bar, I get the following error: > > USING TABLE:logs > > SELECT * FROM search_cache WHERE tablename='logs'Query failed: You have an > error in your SQL syntax; check the manual that corresponds to your MySQL > server version for the right syntax to use near '' at line 1 > > Im getting the error displayed this way because I have it configured to echo > the result. > In the file search.php I see the following query configured: > $sql = "SELECT * FROM ".CACHETABLENAME." WHERE tablename='".$useTable."'"; > 2. When I try to add a user I input the username and password for the new > account and then receive the following message: > > No DB Link > > Im sure these aren't difficult issues to resolve, but im done beating my > head on a wall. Any help would be appreciated. > Stuart > _______________________________________________ > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > https://lists.balabit.hu/mailman/listinfo/syslog-ng > Frequently asked questions at > http://www.campin.net/syslog-ng/faq.html > > > >

1 0

PHP Syslog NG error's
by Stuart Boyle 19 Sep '06

19 Sep '06

Im having a couple of issue's with phpsyslog-ng, I cant seen any threads or posts that help with the problem's im having so I need to post, if im covering old ground, I apologize. Im not well versed with MySQL or PHP, so if you could help please explain in "novice language". 1. When I open the browser and login, I then select "search" from the menu bar, I get the following error: USING TABLE:logs SELECT * FROM search_cache WHERE tablename='logs'Query failed: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '' at line 1 Im getting the error displayed this way because I have it configured to echo the result. In the file search.php I see the following query configured: $sql = "SELECT * FROM ".CACHETABLENAME." WHERE tablename='".$useTable."'"; 2. When I try to add a user I input the username and password for the new account and then receive the following message: No DB Link Im sure these aren't difficult issues to resolve, but im done beating my head on a wall. Any help would be appreciated. Stuart

2 1