- syslog-ng - lists.balabit.hu

Changing a value after a match with patterndb
by Mark Shetka 12 Jan '16

12 Jan '16

Is it possible to substitute a value after a pattern match? For example, in the pattern below if I match the "23" then use value "telnet", but if it is a "22", then "ssh". %SEC_LOGIN-4-LOGIN_FAILED: Login failed [user: test] [Source: 131.212.1.1] [localport: 23] [Reason: Login Authentication Failed] at 10:38:41 CST Thu Jan 23 2014 I am trying to use one pattern to match both cases and we'd prefer to have service names rather than port numbers in our db. -- Mark Shetka Infrastructure Analyst - Network Team Information Technology Systems & Services University of Minnesota - Duluth (218) 726-7682

4 6

how to enable TLS on syslog-ng 3.2.5
by Jim Zapsalis 06 Jan '16

06 Jan '16

Hi guys, How do we enable TLS on OSE version of syslog-ng version 3.2.5? Or is it simpler to enable SSL ? Regards, Jim Zapsalis Senior Platform Engineer, IS Bupa, 33 Exhibition Street, Melbourne, 3000 T +61 3 9937 4028 M 0411 779 710 E jim.zapsalis(a)bupa.com.au<mailto:jim.zapsalis@bupa.com.au> W bupa.com.au/healthandcaring<http://www.bupa.com.au/healthandcaring/index.html> [Social Media Icons-01.png]<https://www.facebook.com/BupaAustralia>[Social Media Icons-02.png]<https://mobile.twitter.com/bupaaustralia>[Social Media Icons-04.png]<https://ea.linkedin.com/company/bupa-australia>[Social Media Icons-03.png]<https://www.youtube.com/user/BupaAustralia> Bupa Australia email disclaimer: The information contained in this email and any attachments is confidential and may be subject to copyright or other intellectual property protection. If you are not the intended recipient, you are not authorized to use or disclose this information, and we request that you notify us by reply mail or telephone and delete the original message from your mail system.

3 3

Proper formatting of incoming Log4j, Jetty events
by Clayton Dukes 16 Dec '15

16 Dec '15

Hi, I'm having trouble with getting "good" fields from Jetty. I found this: http://blog.trifork.com/2010/01/14/logging-to-the-syslog-from-a-java-applic… And had my user configure their log4j with the given recommendation, but the events still come in quite poorly. Our syslog-ng template uses the following template: @${R_UNIXTIME:--}.${R_USEC}\t${HOST:--}\t${PRI:--}\t${PROGRAM:--}\t${MSGONLY}\n"); Some sample events: @1450275007.912996 host001 173 2015-12-16 16:10:07,836 DEBUG [HeartbeatGeneratorTimer] <-- sessionKey=179: out>{"method":"Heartbeat","jsonrpc":"2.0"} As you can see - this event is sending the date as the program name. Another: @1450275007.924140 host002 13 521 <174>1 2015-12-16T16:10:07+02:00 host002 /openam 1000430466 AUTHENTICATION-303 [logRecord@36733 LoginID="id=2565a24-0829-11e2-b614-001e371e7e40,ou=user,dc=opensso,dc=java,dc=net" ContextID="b1de62b91530d5303" IPAddr="10.3.109.13" LogLevel="INFO" Domain="dc=opensso,dc=java,dc=net" ModuleName="ProfileMapper|Mandate|LDAP" HostName="10.3.109.13" LoggedBy="cn=dsameuser,ou=DSAME Users,dc=opensso,dc=java,dc=net" MessageID="AUTHENTICATION-303" NameID="" TIME="2015-12-16 16:10:07"] Logout|service|mailService This one is sending the program name as an integer (521) There are many others, the program names consist of the following (out of about 100k events): - " .0.9"},"parentID" ...80) ame" at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) del" ...e.common.AuthenticationFilter.doFilter(AuthenticationFilter.java ...ee.elion.smarthome.common.AuthenticationFilter.doFilter(AuthenticationFilter.java ...eJspFile(JspServlet.java EST in ,"is_recording" iWeather"},{"id" leRecordID" me" model" ng" ngth" ...rg.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java se,"is_recording" ....servlet.JspServlet.serviceJspFile(JspServlet.java tin Does anyone have a good way to fix this? ______________________________________________________________ Clayton Dukes ______________________________________________________________

2 1

rabbitmq errors
by Al Itchon 15 Dec '15

15 Dec '15

I'm seeing an error in my log indicating that messages aren't being sent to a queue in RabbitMQ. This happens a few times a week. Here's the error: Nov 24 13:29:28 syslog09 syslog-ng[7883]: Network error while inserting into AMQP server; time_reopen='10' I have about 1000 servers sending tcp messages through a call to /dev/tcp to port on the syslog-ng server. It seems messages aren't sent to the queue for about 1 or 2 minutes, and then it resumes sending again without error. Could this be a network issue or something within syslog-ng that's limiting the messages?

4 8

Madhouse apt key is missing
by Clayton Dukes 09 Dec '15

09 Dec '15

Anyone know why https://packages.madhouse-project.org/debian/archive-key.txt is no longer available? ______________________________________________________________ Clayton Dukes ______________________________________________________________

3 5

insider 2015-12: Elasticsearch & Kibana; SCL; Kafka; syslog-ng on Windows; conferences
by Czanik, Péter 03 Dec '15

03 Dec '15

Dear syslog-ng users, This is the 45th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS ESK: Elasticsearch + syslog-ng + Kibana --------------------------------------- An official Elasticsearch destination was a long awaited feature of syslog-ng. The Elasticsearch syslog-ng Kibana (ESK) stack provides high performance log collection, message parsing and filtering on the syslog-ng side and storage and analysis on the Elastic side. Read more about the benefits at https://czanik.blogs.balabit.com/2015/10/how-to-parse-data-with-syslog-ng-s… and https://czanik.blogs.balabit.com/2015/12/elasticsearch-and-syslog-ng-fast-a… The power of SCL ---------------- SCL, the syslog-ng configuration library is a lesser known feature of syslog-ng. It can provide reusable configuration blocks, which help to format messages to a specific format, hide complex configurations, This blog shows how to create a simple SCL through the example of a Loggly (logging as a service provider) destination: https://bazsi.blogs.balabit.com/2015/11/the-power-of-scl-integrating-with-l… Kafka and syslog-ng ------------------- Kafka is a high-throughput distributed messaging system. It was originally developed by LinkedIn as the backbone of a website activity tracking infrastructure and is now used by Netflix, Spotify, Twitter and many other companies for message queuing, log aggregation and stream processing. Read more about how syslog-ng can collect messages from multiple sources and process as well as filter them before forwarding them to Kafka: https://czanik.blogs.balabit.com/2015/11/kafka-and-syslog-ng/ syslog-ng on Windows -------------------- Most Linux admins only consider Windows as a client machine and use syslog-ng Agent for Windows if they need to collect logs from Windows. The following use cases demonstrate why you would want to use syslog-ng Premium Edition server on Windows instead of the syslog-ng Agent for Windows: https://czanik.blogs.balabit.com/2015/11/syslog-ng-on-windows/ syslog-ng @ conferences ----------------------- This autumn, syslog-ng was present at many conferences. I gave presentations about syslog-ng at the annual FSF Hungary and Virtualization & DevOps day conferences (in Hungarian), and participated SUSECon, LISA and Crunch Big Data Conference as an exhibitor. Most questions I received were related to the Elasticsearch destination and also many people were interested in commercial support for syslog-ng. In January I'll present “syslog-ng: from raw data to Big Data” at the Southern California Linux Expo: https://www.socallinuxexpo.org/scale/14x/presentations/syslog-ng-raw-data-b… See you there! NEW RELEASES syslog-ng PE 5F5: https://jluby.blogs.balabit.com/2015/11/16/syslog-ng-premium-edition-5f5-re… syslog-ng OSE 3.7.2: https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.7.2 Your feedback and news tips about the next issue is welcome at documentation(a)balabit.com To read this newsletter on-line, visit: http://insider.blogs.balabit.com/ Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

(no subject)
by Harsha S Aryan 01 Dec '15

01 Dec '15

hi, can any one let me know how to configure syslog-ng -- Rehards, Harsha

2 1

need help to configure
by Harsha S Aryan 01 Dec '15

01 Dec '15

hi, i am looking to configure syslog-ng in my system can any one let me know how to configure and add client (some linux servers and router) -- Rehards, Harsha

2 4

unit test valgrind coverage
by Scheidler, Balázs 24 Nov '15

24 Nov '15

Hi, I've wanted to make our unit tests valgrind clean for a long time, here's where I am at right now, the branch fixes a number of production related bugs as well. Feedback, help appreciated. https://github.com/balabit/syslog-ng/tree/f/unit-test-cleanups I am at 57 successful and 18 failing unit tests so far. -- Bazsi

2 1

log_fetch_limit
by Al Itchon 20 Nov '15

20 Nov '15

I have about 1000 servers sending tcp messages through a call to /dev/tcp to a port on my syslog-ng server. I'm assuming each call to /dev/tcp is a single connection and a single message, so am I correct that the log_fetch_limit should be set to 1 since it applies to each connection? Also, should I bother adjusting the so-rcvbuf? I know the manual mentions that if using UDP, the receive buffer should be increased. I'm using TCP and my message rate is about 250 messages/sec.

2 1