[syslog-ng] loop caused by syslog-ng filter

Sub-Zero Sub-Zero-1 at gmx.at
Thu Mar 20 18:25:51 CET 2008 

Ok, many thanks! 

I´m now at home and therefore not able to test your advices until Tuesday!

I really thought syslog should start a new instance from my script for each
new message.

Bye, Adam

-----Ursprüngliche Nachricht-----
Von: syslog-ng-bounces at lists.balabit.hu
[mailto:syslog-ng-bounces at lists.balabit.hu] Im Auftrag von Fegan, Joe
Gesendet: Donnerstag, 20. März 2008 16:33
An: Syslog-ng users' and developers' mailing list
Betreff: Re: [syslog-ng] loop caused by syslog-ng filter

> The loop starts when syslog-ng recognices the first
> machting string [Priority: 1] and loops till I stop syslog-ng!!!
>
> Sensor1:~# /etc/init.d/syslog-ng stop
> Stopping system logging: syslog-ng.
> Sensor1:~# tail -f /tmp/schrott
> AAA
> Do 20. Mär 14:29:12 CET 2008
> AAA
> Do 20. Mär 14:29:12 CET 2008
> AAA
> Do 20. Mär 14:29:12 CET 2008
> AAA
> Do 20. Mär 14:29:12 CET 2008
> AAA
> Do 20. Mär 14:29:12 CET 2008


Here is the explanation:

Your script appends to /tmp/schrott and exits. Syslog-ng restarts it.
Your script appends to /tmp/schrott and exits. Syslog-ng restarts it.
Your script appends to /tmp/schrott and exits. Syslog-ng restarts it.
Your script appends to /tmp/schrott and exits. Syslog-ng restarts it.
Your script appends to /tmp/schrott and exits. Syslog-ng restarts it.

... and so on, forever ...

The reason is that the script is not obeying the rules for a syslog-ng
destination, as previously stated.


-----Original Message-----
From: syslog-ng-bounces at lists.balabit.hu
[mailto:syslog-ng-bounces at lists.balabit.hu] On Behalf Of Fegan, Joe
Sent: 20 March 2008 15:11
To: Syslog-ng users' and developers' mailing list
Subject: Re: [syslog-ng] loop caused by syslog-ng filter

Hi Adam,

Please listen to the advice you're being given. From your examples, I think
you believe that syslog-ng starts a new instance of your script for each new
message. This is not true. Syslog-ng starts *one* instance of your script
during its own startup, and it expects that one instance to stay alive for a
long time. The script must read lines from its stdin *in a loop* until
syslog-ng hangs up the connection (which will happen when syslog-ng is
shutting down). Each line is an individual message.

Both of your example scripts exit immediately. You can't do that. Syslog-ng
will automatically restart any destination script that exits, so your
scripts will be restarted over and over and over and over because they keep
exiting. Eventually syslog-ng will go crazy doing this.

As Christian said, a destination script needs a "read" loop like this
example:

#!/bin/bash
while
  read MSG
do
  # Process the message $MSG. Do whatever you like with it.
  # Email it to someone. Whatever. Here we put it in a file.

  echo $MSG >> /tmp/schrott

  # We're finished with this message, wait for the next one.
done
# Read returned eof. This means "no more messages". Exit.
exit 0


Christian's tips for testing destination scripts are also good. Run the
script from the command line yourself. Type in messages like the ones you
expect syslog-ng to send it (cut/paste them from /var/log/messages or
wherever). Check that the script does what you expect with each message and
that it waits for the next one, does *not* exit immediately. When it is
working interactively then you can hook it up to syslog-ng.

Joe.


-----Original Message-----
From: syslog-ng-bounces at lists.balabit.hu
[mailto:syslog-ng-bounces at lists.balabit.hu] On Behalf Of JUNG, Christian
Sent: 20 March 2008 14:22
To: Syslog-ng users' and developers' mailing list
Subject: Re: [syslog-ng] loop caused by syslog-ng filter

Okay. Don't know if I get the things right, but:

All scripts you've mailed to the list write something to a file or send a
mail if they're called and terminate directly after that action. syslog-ng
will respawn (neu starten) them directly.

The program started by the program destination should not terminate itself.
It has to listen on STDIN for a log message given from syslog-ng. Afterwards
it can do something useful and then it has to listen for the next message.

In a shell script you can do this with a "while read LINE; do ...; done".

Try this:

---8<---
#!/bin/bash

while read LINE; do
        echo $(date) $LINE >> /tmp/schrott
done
---8<---

Execute this script on the command line, enter some random stuff and look
into /tmp/schrott:

user at box:~> ./test-script
bla bla bla
bla

Terminate this script by pressing CTRL-D. You should see something like this
in /tmp/schrott:

Thu Mar 20 15:13:16 CET 2008 bla bla bla
Thu Mar 20 15:13:18 CET 2008 bla

If not your script doesn't work :-) See for typos.

If this works, put it in your syslog-ng conf. It should do the same (only
prio 1 messages should be visible with the date prepended).

If this works well, try this script first on the command line:

---8<---
#!/bin/bash

while read LINE; do
        mail -s "High Priority Snort Alert"  Sub-Zero at xxx.de <<-EOF
        Alert, Priority 1
        $LINE
        EOF
done
---8<---

You should receive for every given input line exactly one mail.

If this works put it in your syslog-ng.conf. Now you should be done.

The thread you mentioned is about two running scripts where only one should
run. This shouldn't be the case here. But you can have a look at the output
of "ps fax". If you see multiple processes under syslog-ng then you might
have the problem.


bye
Chris

> -----Original Message-----
> From: syslog-ng-bounces at lists.balabit.hu
> [mailto:syslog-ng-bounces at lists.balabit.hu]On Behalf Of Adam Richter
> Sent: Thursday, March 20, 2008 2:46 PM
> To: Syslog-ng users' and developers' mailing list
> Subject: Re: [syslog-ng] loop caused by syslog-ng filter
>
>
> Hi!
>
> Not working! Syslog-ng filters for exactly the string: [Priority: 1]
> and not as it is piped by the mail script: #priority 1# note the ":" !
> Anyway I used your script --> same fault!!!
> I have also used following script:
>
>
> #!/bin/sh
>
> echo AAA >> /tmp/schrott
> date >> /tmp/schrott
>
> There is no output like [Priority: 1]!!!
> Then I did following: tail -f /tmp/schrott and got a loop
> too! The loop starts when syslog-ng recognices the first
> machting string [Priority: 1] and loops till I stop syslog-ng!!!
>
>
> Sensor1:~# /etc/init.d/syslog-ng stop
> Stopping system logging: syslog-ng.
> Sensor1:~# tail -f /tmp/schrott
> AAA
> Do 20. Mär 14:29:12 CET 2008
> AAA
> Do 20. Mär 14:29:12 CET 2008
> AAA
> Do 20. Mär 14:29:12 CET 2008
> AAA
> Do 20. Mär 14:29:12 CET 2008
> AAA
> Do 20. Mär 14:29:12 CET 2008
>
>
> I think it has something in common with this thread:
>
> https://lists.balabit.hu/pipermail/syslog-ng/2006-October/009454.html
>
> Any other ideas? It´s very important!
>
> German: Es handelt sich hier um meine Abschlussprüfung, und
> dies ist der letzte Fehler der Auftritt, ansonsten läuft das Projekt.
>
> bye, Adam / Sub-Zero !
>
>
> -------- Original-Nachricht --------
> > Datum: Thu, 20 Mar 2008 10:32:31 +0100
> > Von: "JUNG, Christian" <christian.jung at saarstahl.com>
> > An: "Syslog-ng users\' and developers\' mailing list"
> <syslog-ng at lists.balabit.hu>
> > Betreff: Re: [syslog-ng] loop caused by syslog-ng filter
>
> > Hi Adam,
> >
> > syslog-ng does the right thing :-).
> >
> > It starts the program/script once and pipes on STDIN every
> log-message
> > which matches the filter.
> >
> > If your script is started, it will call mail and pipe
> "Alert, priority 1"
> > to its STDIN and then exits. syslog-ng sees this and
> restarts it (version
> > 2.0 or higher behave that way, see
> >
> <http://www.balabit.com/dl/html/syslog-ng-admin-guide_en.html/
> ch09s02.html#reference_destination_program>).
> >
> > For you purpose this would be better:
> >
> > ---8<---
> > #!/bin/bash
> >
> > while read LINE; do
> >     cat <<-EOF | mail -s "High Priority Snort Alert"
> Sub-Zero at xxx.de
> >     Alert, Priority 1
> >     $LINE
> >     EOF
> > done
> > ---8<---
> >
> >
> > bye
> > Chris
> >
> > >
> > > -----Original Message-----
> > > From: syslog-ng-bounces at lists.balabit.hu
> > > [mailto:syslog-ng-bounces at lists.balabit.hu]On Behalf Of
> Adam Richter
> > > Sent: Thursday, March 20, 2008 9:23 AM
> > > To: syslog-ng at lists.balabit.hu
> > > Subject: [syslog-ng] loop caused by syslog-ng filter
> > >
> > >
> > > Hi!
> > >
> > > First off, sorry for my poor english!
> > > I have a problem with a loop caused by syslog-ng v. 2.0.8.  I
> > > have set up Snort as an IDS System. Snort writes its messages
> > > in unified-format to /var/log/snort/snort.alert and
> > > /var/log/snort/snort.log. There are two Barnyard processes
> > > which read the unified files and convert it to messages that
> > > syslog and MySQL understand. Syslog-ng  writes the messages
> > > to /var/log/auth.log. All this is working fine. Now, I want
> > > to set up a filter for Priority 1 alerts. This alert should
> > > be send to the Administrator.
> > >
> > > I used following filter for syslog-ng:
> > >
> > > source src {unix-stream("/dev/log"); internal();};
> > > destination email{program("/usr/local/bin/alert_mail.sh");};
> > > filter high {match("[Priority: 1]");};
> > > log {source(src);filter(high); destination(email);};
> > >
> > >
> > > The alert_mail.sh:
> > >
> > > #!/bin/sh
> > > cat << EOF | mail -s "High Priority Snort Alert" Sub-Zero at xxx.de
> > > Alert, Priority 1
> > > EOF
> > >
> > >
> > > Then I use Nessus to cause some alerts with Priority 1. I can
> > > see 4 alerts with the Priority 1 with BASE and in
> /var/log/auth.log.
> > >
> > > Syslog-ng recognises the alert with Priority 1 and activates
> > > the script /usr/local/bin/alert_mail.sh
> > >
> > > All this is working, but the script is restarted by syslog-ng
> > > again an again.
> > >
> > > Extract from /var/log/messages:
> > >
> > > Mar 18 15:46:16 Sensor1 syslog-ng[5191]: Child program
> > > exited, restarting; cmdline='/usr/local/bin/alert_mail.sh ',
> > > status='0'
> > > Mar 18 15:46:16 Sensor1 syslog-ng[5191]: Starting destination
> > > program; cmdline='/usr/local/bin/alert_mail.sh '
> > > Mar 18 15:46:16 Sensor1 syslog-ng[5191]: Closing log writer
> > > fd; fd='11'
> > > Mar 18 15:46:16 Sensor1 syslog-ng[5191]: Child program
> > > exited, restarting; cmdline='/usr/local/bin/alert_mail.sh ',
> > > status='0'
> > > Mar 18 15:46:16 Sensor1 syslog-ng[5191]: Starting destination
> > > program; cmdline='/usr/local/bin/alert_mail.sh '
> > > Mar 18 15:46:16 Sensor1 syslog-ng[5191]: Child program
> > > exited, restarting; cmdline='/usr/local/bin/alert_mail.sh ',
> > > status='0'
> > > Mar 18 15:46:16 Sensor1 syslog-ng[5191]: Starting destination
> > > program; cmdline='/usr/local/bin/alert_mail.sh '
> > > ...
> > > Mar 18 15:42:00 Sensor1 syslog-ng[17354]: Child program
> > > exited, restarting; cmdline='/usr/local/bin/alert_mail.sh ',
> > > status='256'
> > > Mar 18 15:42:00 Sensor1 syslog-ng[17354]: Starting
> > > destination program; cmdline='/usr/local/bin/alert_mail.sh '
> > > Mar 18 15:42:00 Sensor1 syslog-ng[17354]: Child program
> > > exited, restarting; cmdline='/usr/local/bin/alert_mail.sh ',
> > > status='256'
> > > Mar 18 15:42:00 Sensor1 syslog-ng[17354]: Starting
> > > destination program; cmdline='/usr/local/bin/alert_mail.sh '
> > > Mar 18 15:42:00 Sensor1 syslog-ng[17354]: Child program
> > > exited, restarting;
> > >
> > > ...
> > >
> > >
> > > I get thousands of mails per minute till I stop syslog-ng.
> > >
> > > Output of /var/log/auth.log(so y see that syslog-ng writes
> > > snort/barnyard messages correctly to auth.log):
> > >
> > > Mar 19 13:56:54 src at Sensor1 barnyard: [1:1394:8] SHELLCODE
> > > x86 NOOP [Classification: Executable code was detected]
> > > [Priority: 1] {UDP} 172.25.1.152:4758 -> 172.28.100.10:137
> > > Mar 19 13:57:13 src at Sensor1 barnyard: [1:1446:8] SMTP vrfy
> > > root [Classification: Attempted Information Leak] [Priority:
> > > 2] {TCP} 172.25.1.152:1085 -> 172.28.100.10:25
> > > Mar 19 13:57:13 src at Sensor1 barnyard: [1:660:11] SMTP expn
> > > root [Classification: Attempted Information Leak] [Priority:
> > > 2] {TCP} 172.25.1.152:1085 -> 172.28.100.10:25
> > > Mar 19 13:57:22 src at Sensor1 barnyard: [1:12626:2] Snort Alert
> > > [1:12626:0] [Classification: Decode of an RPC Query]
> > > [Priority: 2] {UDP} 172.25.1.152:1146 -> 172.28.100.10:111
> > > Mar 19 13:57:22 src at Sensor1 barnyard: [1:585:9] RPC portmap
> > > sadmind request UDP [Classification: Decode of an RPC Query]
> > > [Priority: 2] {UDP} 172.25.1.152:1146 -> 172.28.100.10:111
> > > Mar 19 13:57:24 src at Sensor1 barnyard: [1:566:6] POLICY
> > > PCAnywhere server response [Classification: Misc activity]
> > > [Priority: 3] {UDP} 172.25.1.152:1155 -> 172.28.100.10:5632
> > > Mar 19 15:11:27 src at Sensor1 barnyard: [122:1:0] portscan: TCP
> > > Portscan [Classification: Unknown] [Priority: 3] {PROTO255}
> > > 172.25.1.152 -> 172.28.100.10
> > > Mar 19 15:11:58 src at Sensor1 barnyard: [1:1420:13] SNMP trap
> > > tcp [Classification: Attempted Information Leak] [Priority:
> > > 2] {TCP} 172.25.1.152:4482 -> 172.28.100.10:162
> > > Mar 19 15:11:58 src at Sensor1 barnyard: [1:1418:13] SNMP
> > > request tcp [Classification: Attempted Information Leak]
> > > [Priority: 2] {TCP} 172.25.1.152:4482 -> 172.28.100.10:161
> > > Mar 19 15:12:05 src at Sensor1 barnyard: [1:1421:13] SNMP
> > > AgentX/tcp request [Classification: Attempted Information
> > > Leak] [Priority: 2] {TCP} 172.25.1.152:4482 -> 172.28.100.10:705
> > > Mar 19 15:12:16 src at Sensor1 barnyard: [122:1:0] portscan: TCP
> > > Portscan [Classification: Unknown] [Priority: 3] {PROTO255}
> > > 172.25.1.152 -> 172.28.100.10
> > > Mar 19 15:12:19 src at Sensor1 barnyard: [1:1394:8] SHELLCODE
> > > x86 NOOP [Classification: Executable code was detected]
> > > [Priority: 1] {UDP} 172.25.1.152:137 -> 172.28.100.10:137
> > > Mar 19 15:12:20 src at Sensor1 barnyard: [1:1394:8] SHELLCODE
> > > x86 NOOP [Classification: Executable code was detected]
> > > [Priority: 1] {UDP} 172.25.1.152:137 -> 172.28.100.10:137
> > > Mar 19 15:12:22 src at Sensor1 barnyard: [1:1394:8] SHELLCODE
> > > x86 NOOP [Classification: Executable code was detected]
> > > [Priority: 1] {UDP} 172.25.1.152:137 -> 172.28.100.10:137
> > >
> > >
> > > I think it has something in common with this topic:
> > >
> > >
https://lists.balabit.hu/pipermail/syslog-ng/2006-October/009454.html
> >
> > Thanks in advance!
> >
> > Sub-Zero
> >
> >
> > --
> > GMX startet ShortView.de. Hier findest Du Leute mit Deinen Interessen!
> > Jetzt dabei sein: http://www.shortview.de/?mc=sv_ext_mf@gmx
> > ______________________________________________________________
> > ________________
> > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
>
____________________________________________________________________________
__
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>

--
Psst! Geheimtipp: Online Games kostenlos spielen bei den GMX Free Games!
http://games.entertainment.gmx.net/de/entertainment/games/free
____________________________________________________________________________
__
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation:
http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.campin.net/syslog-ng/faq.html

____________________________________________________________________________
__
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation:
http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.campin.net/syslog-ng/faq.html

____________________________________________________________________________
__
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation:
http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.campin.net/syslog-ng/faq.html

____________________________________________________________________________
__
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation:
http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.campin.net/syslog-ng/faq.html

More information about the syslog-ng mailing list