- syslog-ng - lists.balabit.hu

Issue about using perl script in Receiving messages from external applications
by Jian Zhu 10 Aug '12

10 Aug '12

Hello, Could anyone please help me on this below issue? my syslog-ng.conf file look like below: The test.sh script work fine, I can see it's output in the test.sh.log file. But the test.pl doesn't work. the test.sh and test.pl have the same function of printing something to STDOUT. ------------------------------ ------------------------------------------------------------------------------------------------------------------- @version: 3.0 #Default configuration file for syslog-ng. # # For a description of syslog-ng configuration file directives, please read # the syslog-ng Administrator’s guide at: # # http://www.balabit.com/dl/html/syslog-ng-admin-guide_en.html/bk01-toc.html # options { # Number of syslog lines stored in memory before being written to files flush_lines (0); log_fifo_size (2048); create_dirs (yes); perm (0640); dir_perm (0750); }; source ppp { program("/root/perl/test.pl"); }; destination ppp { file ("/var/log/syslog-ng/$HOST/perl.pl.log" create_dirs(yes)); }; log { source(ppp); destination(ppp); }; source sss { program("/root/shell/test.sh"); }; destination sss { file ("/var/log/syslog-ng/$HOST/test.sh.log" create_dirs(yes)); }; log { source(sss); destination(sss); }; ------------------------------------------------------------------------------------------------------------------------------------------------- -- *Thanks & Best Regards* *James Zhu* *Mobile**:* *+86 18**6 8482 2818* *Email: *zhujian0805(a)gmail.com

1 0

Query On configuring Centralized Audit server with Auditd daemon
by Koresh... 09 Aug '12

09 Aug '12

Hi Folks, Need your help ! Want to configure a centralized Audit server (Currently the centralized server is running Octopussy Web interface, which receives logs from remote hosts by Rsyslog ). The challenge and confusion here is .. all my linux clients are configured with syslog-ng and the daemon is sending all the system logs and kernel logs like messages,secure,cron logs etc ... with out any trouble. The problem is the syslog-ng daemon is not able to send the auidtd logs (/var/log/audit.log) to the Rsyslog server, Hence request your help to guide me how to setup the syslog-ng to forward the audit.log to the remote Rsyslog server. It would be great if i can get client side and server side configuration guidelines. -- Thanks in Advance - Koresh

3 4

syslog-ng service not starting
by James McDonald 08 Aug '12

08 Aug '12

I returned from being out of the office for week and realized something was "off" with syslog-ng. I was unable to restart the service. I did a 'service syslog-ng status' and it says it's stopped. When I try 'service syslog-ng start' I get the following error: Error binding socket; addr='AF_INET(0.0.0.0:443)', error='Address already in use (98)' Error initializing source driver; source='inputs', id='inputs#3' Error initializing message pipeline; There is only instance of syslog-ng installed on the server but I did a reboot to try and clear any conflicting settings, but I still get the same error. Can someone help me clear this error and get the service started again? Thanks, James McDonald System Administrator RHCSA, Security+ jmcdonald(a)lce.com

5 8

query about dropped messages...
by Vijendra Agarwal 06 Aug '12

06 Aug '12

Hi, I am using syslog-ng 3.1.4 open source edition. my setup details are.. I have Log application running on MC1 with syslog-ng configured as client on it. My log application is communicating witg syslog-ng on unix-stream socket. I have machine M2 where my syslog-ng server is running. My syslog-ng client is coomunicating with syslog-ng server on TCP stream. It writes all the log messages received from my log application to the /var/log/messages file. My log application sends bulk of message around 600000 p/s , I have noticed from the syslog-statistics that the syslog-ng client dropped some log messages. I have a query here Can I store the dopped messages to some file by doing some configration setting? Or Is there a configyration setting by which I can avoide the dropping of messages by syslog-ng client? Kindly help me out to resolve the issue. Thanks Vijendra. -- Thanks Vij M No: 9960771025.

1 0

Syslog-ng OSE 3.3.5 - High CPU usage
by satish pattar 31 Jul '12

31 Jul '12

Hi All, I am using syslog-ng OSE & seeing a high CPU usage ? Could you please point out if this could be a config/rules issue or syslog-ng internal issue. CPU usage in sorted order : 0.0 11.9 16.6 22.2 25.0 25.0 25.0 25.0 33.3 33.3 50.0 ------------------------------------------ # syslog-ng --version syslog-ng 3.3.5 Installer-Version: 3.3.5 Revision: Compile-Date: Apr 20 2012 04:47:44 Default-Modules: afsocket,affile,afprog,afuser,basicfuncs,syslogformat,dbparser Available-Modules: syslogformat,dbparser,afuser,basicfuncs,afmongodb,convertfuncs,afprog,dummy,afsocket,confgen,affile,csvparser Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off Enable-Pcre: off ----------- Here is the code snippet to send syslog messages to syslogger for(i=0; i<500;i++) { // this is the message format we are following syslog(LOG_INFO, "%s, %s, %d, %s, %s, %s %d, %s", "00:00:18","XYZ",getpid(),"Info","xyz.c", "Func1", 100, "WelCome to Syslog" ); } --------------------- # cat /proc/cpuinfo Processor : ARMv7 Processor rev 10 (v7l) processor : 0 BogoMIPS : 1988.29 processor : 1 BogoMIPS : 1988.29 Features : swp half thumb fastmult vfp edsp neon vfpv3 CPU implementer : 0x41 CPU architecture: 7 CPU variant : 0x2 CPU part : 0xc09 CPU revision : 10 ------------------------------ syslog-ng-confg: @version: 3.3 ###### # options options { # disable the chained hostname format in logs # (default is enabled) chain_hostnames(0); # the time to wait before a died connection is re-established # (default is 60) time_reopen(10); # the time to wait before an idle destination file is closed # (default is 60) time_reap(360); # the number of lines buffered before written to file # you might want to increase this if your disk isn't catching with # all the log messages you get or if you want less disk activity flush_lines(5); flush_timeout(500); # the number of lines fitting in the output queue log_fifo_size(10000); # enable or disable directory creation for destination files create_dirs(yes); # default owner, group, and permissions for log files # (defaults are 0, 0, 0600) #owner(root); group(root); perm(0640); frac_digits(4); # Override the global timestamp format #(set in the global ts_format() parameter) for the specific destination. ts_format(iso); # default owner, group, and permissions for created directories # (defaults are 0, 0, 0700) #dir_owner(root); dir_perm(0755); # enable or disable DNS usage # syslog-ng blocks on DNS queries, so enabling DNS may lead to # a Denial of Service attack # (default is yes) use_dns(no); # maximum length of message in bytes # this is only limited by the program listening on the /dev/log Unix # socket, glibc can handle arbitrary length log messages, but -- for # example -- syslogd accepts only 1024 bytes # (default is 2048) #log_msg_size(2048); # print statistic information only once every hour (it is not really needed) stats_freq(3600); }; ###### # sources # all known message sources source s_all { # message generated by Syslog-NG internal(); # standard Linux log source (this is the default place for the syslog() # function to send logs to) unix-dgram("/dev/log" program_override("mylogger: ")); # messages from the kernel file("/proc/kmsg" program_override("kernel: ")); # use the following line if you want to receive remote UDP logging messages # (this is equivalent to the "-r" syslogd flag) # udp(); }; ######## #templates #template t_default { template("${ISODATE} ${HOST} ${MSGHDR}${MSG}\n"); template_escape(no); }; #template t_logtty { template("${MSG}\n"); template_escape(no); }; template t_restmsg_notempty { template("${PROGRAM} ${date} ${cname}[${processId}] ${type} (${source}:${line}) ${Restmsg}\n"); }; template t_restmsg_empty { template("${PROGRAM} ${FACILITY} ${LEVEL} ${MSG}\n"); }; ###### # destinations destination d_log_notempty { file("/var/my_log" template(t_restmsg_notempty)); }; #destination d_log_notempty { file("/dev/console" template(t_restmsg_notempty)); }; destination d_log_empty { file("/var/my_log" template(t_restmsg_empty)); }; destination authlog { file("/var/log/auth.log"); }; destination debug { file("/var/log/debug"); }; destination messages { file("/var/log/messages"); }; destination _syslog { file("/var/log/syslog"); }; destination cron { file("/var/log/cron.log"); }; destination daemon { file("/var/log/daemon.log"); }; destination lpr { file("/var/log/lpr.log"); }; destination user { file("/var/log/user.log"); }; destination uucp { file("/var/log/uucp.log"); }; destination mail { file("/var/log/mail.log"); }; destination kern { file("/var/log/kern.log"); }; destination mailinfo { file("/var/log/mail.info"); }; destination mailwarn { file("/var/log/mail.warn"); }; destination mailerr { file("/var/log/mail.err"); }; destination newscrit { file("/var/log/news/news.crit"); }; destination newserr { file("/var/log/news/news.err"); }; destination newsnotice { file("/var/log/news/news.notice"); }; ###### # filters # all messages from the auth and authpriv facilities filter f_restmsg_notempty {match(".+" value("Restmsg"));}; filter f_restmsg_empty {not match(".+" value("Restmsg"));}; # all messages except from the auth and authpriv facilities filter f_auth { facility(auth); }; filter f_authpriv { not facility(auth, authpriv); }; #filter f_debug { level(notice) and not facility(auth, authpriv, news, mail); }; filter f_emergency { level(emerg); }; filter f_info { level(info); }; filter f_notice { level(notice); }; filter f_warn { level(warn); }; filter f_crit { level(crit); }; filter f_err { level(err); }; #filter f_syslog { not facility(authpriv, mail ); }; filter f_syslog { program(syslog-ng); }; filter f_cron { facility(cron); }; filter f_daemon { facility(daemon); }; filter f_kern { facility(kern); }; filter f_mail { facility(mail); }; filter f_user { facility(user); and level(error) }; filter f_uucp { facility(uucp); }; filter f_news { facility(news); }; filter f_messages { level(info..warn) and not facility(auth, authpriv, mail, news); }; filter f_all_info { not match("Info", value("type"), type("string")); }; filter f_all_generic{ not match("Generic", value("cname"), type("string")); }; parser p_db {db_parser( file("/etc/syslog-ng/rules/rules.xml") );}; log {source(s_all);parser(p_db);filter(f_restmsg_notempty);destination(d_log_notempty); }; #log { source(s_all); filter(f_authpriv); destination(authlog); }; log { source(s_all); filter(f_syslog); destination(_syslog); }; log { source(s_all); filter(f_cron); destination(cron); }; log { source(s_all); filter(f_daemon); destination(daemon); }; log { source(s_all); filter(f_kern); destination(kern); }; log { source(s_all); filter(f_mail); destination(mail); }; log { source(s_all); filter(f_user); destination(user); }; #log { source(s_all); filter(f_debug); destination(debug); }; ------------------------------------------------------------------------------------- rules.xml: <patterndb version='4' pub_date='2012-02-17'> <ruleset name='root' id='123456678'> <pattern>mylogger</pattern> <rules> <rule provider='test' id='182437592347598' class='system'> <patterns> <pattern>@ESTRING:date:,@ @ESTRING:cname:,@ @ESTRING:processId:,@ @ESTRING:type:,@ @ESTRING:source:,@ @ESTRING:line:,@ @ANYSTRING:Restmsg: @</pattern> </patterns> </rule> </rules> </ruleset> </patterndb> --------------------------------------------------------------------------------------------- Thanks, pattarsatish

2 1

Installation Error
by kibirango moses 28 Jul '12

28 Jul '12

Hullo users, I am installing syslog-ng ver.3.3.5 on slackware linux 13.1 and i am getting the errors below, when i do a make usr/lib64/gcc/x86_64-slackware-linux/4.4.4/../../../../x86_64-slackware-linux/bin/ld: /usr/local/lib64/libssl.a(s23_srvr.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC /usr/local/lib64/libssl.a: could not read symbols: Bad value Could somebody helpme with a fix to the problem thanx in advnace

2 1

Logging To MongoDB and specifying data types
by Michael Dillon 27 Jul '12

27 Jul '12

I'm using the syslog-ng windows client to forward IIS logs to syslog-ng. I'm using the CSV parser to split it into pairs and pick and choose my pairs. I'm not able to figure out how to insert a field from these pairs (or any others) into MongoDB as a date time field. They are being stored as strings. Also I'd like to store quite a few of the other columns/elements as integers. Obviously I could run a job to do conversions after they are inserted into the database I'd just hate to setup a rube golberg machine if I don't have to. -- Michael Dillon

2 1

Able to connect but not logging
by kibirango moses 27 Jul '12

27 Jul '12

Hullo Users; I am using syslog-ng 3.3.4 and my client is able to connect but not logging to the central logging server. my syslog-ng.conf on server side is @version: 3.3 # Default syslog-ng configuration for Slackware # For info about the format of this file, see "man syslog-ng.conf" # Converted from sysklogd (syslog.conf) by mario(a)slackverse.org options { flush_lines (0); time_reopen (10); time_reap(360); # log_fifo_size (10240); # log_msg_size (8192); chain_hostnames (no); use_dns (no); use_fqdn (no); create_dirs (yes); keep_hostname (yes); owner("root"); group("root"); dir_owner(root); # dir_group(logs); dir_perm(0750); perm(0640); stats_freq (3600); check_hostname (yes); dns_cache (yes); threaded(yes); mark_freq(600); }; source system { internal(); unix-stream("/dev/log"); udp(); tcp(max_connections(100)); }; filter f_messages { level(info,notice) and not facility(authpriv,cron,mail,news); }; filter f_syslog { level(warn..emerg) and not facility(authpriv,cron,mail,news); }; filter f_debug { level(debug); }; filter f_authpriv { facility(authpriv); }; filter f_cron { facility(cron); }; filter f_mail { facility(mail); }; filter f_emerg { level(emerg); }; filter f_uucp { facility(uucp); }; destination d_messages { file("/var/log/messages"); }; destination d_syslog { file("/var/log/syslog"); }; destination d_debug { file("/var/log/debug"); }; destination d_secure { file("/var/log/secure"); }; destination d_cron { file("/var/log/cron"); }; destination d_maillog { file("/var/log/maillog"); }; destination d_usertty { usertty("*"); }; destination d_spooler { file("/var/log/spooler"); }; # Log anything 'info' or higher, but lower than 'warn'. # Exclude authpriv, cron, mail, and news. These are logged elsewhere. log { source(system); filter(f_messages); destination(d_messages); }; # Log anything 'warn' or higher. # Exclude authpriv, cron, mail, and news. These are logged elsewhere. log { source(system); filter(f_syslog); destination(d_syslog); }; # Debugging information is logged here. log { source(system); filter(f_debug); destination(d_debug); }; # Private authentication message logging: log { source(system); filter(f_authpriv); destination(d_secure); }; # Cron related logs: log { source(system); filter(f_cron); destination(d_cron); }; # Mail related logs: log { source(system); filter(f_mail); destination(d_maillog); }; # Emergency level messages go to all users: log { source(system); filter(f_emerg); destination(d_usertty); }; # This log is for news and uucp errors: log { source(system); filter(f_uucp); destination(d_spooler); }; # Uncomment this to see kernel messages on the console. #filter f_kern { facility(kern); }; #destination d_console { file("/dev/console"); }; #log { source(system); filter(f_kern); destination(d_console); }; # Uncomment these if you'd like INN to keep logs on everything. # You won't need this if you don't run INN (the InterNetNews daemon). #filter f_news_crit { facility(news) and level(crit); }; #filter f_news_err { facility(news) and level(err); }; #filter f_news_notice { facility(news) and level(notice); }; #destination d_news_crit { file("/var/log/news/news.crit"); }; #destination d_news_err { file("/var/log/news/news.err"); }; #destination d_news_notice { file("/var/log/news/news.notice"); }; #log { source(system); filter(f_news_crit); destination(d_news_crit); }; #log { source(system); filter(f_news_err); destination(d_news_err); }; #log { source(system); filter(f_news_notice); destination(f_news_notice); }; # Remote logging source s_remote { tcp(ip(0.0.0.0) port(514)); udp(ip(0.0.0.0) port(514)); }; destination logpile { file("/var/log/$HOST/$YEAR/$MONTH/$FACILITY.$YEAR$MONTH$DAY" owner(root) group(root) perm(0600) create_dirs(yes) dir_perm(0700)); }; log { source(system); destination(logpile); }; Cleint syslog-ng.conf I just added the lines belo to the server file above # Remote logging destination logpile { file("/var/log/$HOST/$YEAR/$MONTH/$FACILITY.$YEAR$MONTH$DAY" owner(root) group(root) perm(0600) create_dirs(yes) dir_perm(0700)); }; destination remote { tcp("196.43.133.98" port(514)); }; log { source(system); destination(logpile); }; log { source(system); destination(remote); }; testing for connection root@client:/# lsof -i | grep syslog-ng syslog-ng 18339 root 9u IPv4 37485820 0t0 UDP *:syslog syslog-ng 18339 root 10u IPv4 37485822 0t0 TCP *:shell (LISTEN) syslog-ng 18339 root 11u IPv4 37485824 0t0 TCP client:48305-> syslog server:shell (ESTABLISHED) I am not seeing any logging being registered . How can i solve this and where have i gone wrong. Thanx i advance

2 1

[Bug 187] New: Syslog-ng truncates SDATA PARAM VALUE to 256 chars
by bugzilla＠bugzilla.balabit.com 27 Jul '12

27 Jul '12

https://bugzilla.balabit.com/show_bug.cgi?id=187 Summary: Syslog-ng truncates SDATA PARAM VALUE to 256 chars Product: syslog-ng Version: 3.2.x Platform: All OS/Version: All Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: cloun-rulez(a)yandex.ru QAContact: cloun-rulez(a)yandex.ru Type of the Report: enhancement Estimated Hours: 0.0 As you can see there are no limitations of SDATA PARAM VALUE length in rfc5424 http://tools.ietf.org/html/rfc5424 As I have saw in tests/unit/test_msgparse.c this is default behavior of syslog-ng. // too long sdata value gets truncated testcase("<132>1 2006-10-29T01:59:59.156+01:00 mymachine evntslog - - [a i=\"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\"] An application event log entry...", LP_SYSLOG_PROTOCOL, NULL, 132, // pri 1162083599, 156000, 3600, // timestamp (sec/usec/zone) "mymachine", // host "evntslog", //app "An application event log entry...", // msg "[a i=\"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\"]", //sd_str "",//processid "",//msgid expected_sd_pairs_test_5b ); I'm trying to increase limit of truncation but still haven't success. Need help. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 5

non-UTF-8 data in mongodb
by Lucas, Sascha 25 Jul '12

25 Jul '12

Hi, I was looking for a way to have some log-rotate functionality (move logs older then x into another collection). I heard of TTL collections, which is fine for deletion. To keep collections small, I tried following mongo-shell code: for( var c = db.messages.find({DATE: {$lt: "1342688698"}}); c.hasNext(); ) { db.messages.old.save(c.next()); } Being totally new to mongodb, I don’t know if this is the right solution, however I got the following error: Mon Jul 23 12:30:34 decode failed. probably invalid utf-8 string [opx2 opx2-dispatch.exe(8100) 2 OPX-20001 Attempt to do an array operation on #26\META-SUPER-HYPER-� which is not an array.] Mon Jul 23 12:30:34 why: TypeError: malformed UTF-8 character sequence at offset 98 Mon Jul 23 12:30:34 Error: invalid utf8 shell/collection.js:128 Asking on mongodb-user Tad Marshall explained that running mongodb with --objcheck prevents this problem. The mogodb config file has following comment: # Inspect all client data for validity on receipt (useful for # developing drivers) #objcheck = true Is the syslog-ng shipped driver the root cause? Should syslog-ng utf8-encode the strings? Is this a bug, or do I miss something? TIA, Sascha. Vorsitzender des Aufsichtsrates: Ralf Hiltenkamp Geschäftsführung: Michael Krüger (Sprecher), Stephan Drescher Sitz der Gesellschaft: Halle/Saale Registergericht: Amtsgericht Stendal | Handelsregister-Nr. HRB 208414 UST-ID-Nr. DE 158253683 Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Empfänger sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail oder des Inhalts dieser Mail sind nicht gestattet. Diese Kommunikation per E-Mail ist nicht gegen den Zugriff durch Dritte geschützt. Die GISA GmbH haftet ausdrücklich nicht für den Inhalt und die Vollständigkeit von E-Mails und den gegebenenfalls daraus entstehenden Schaden. Sollte trotz der bestehenden Viren-Schutzprogramme durch diese E-Mail ein Virus in Ihr System gelangen, so haftet die GISA GmbH - soweit gesetzlich zulässig - nicht für die hieraus entstehenden Schäden.

2 2