[syslog-ng] loop caused by syslog-ng filter
Fegan, Joe
Joe.Fegan at hp.com
Thu Mar 20 16:33:10 CET 2008
> The loop starts when syslog-ng recognices the first
> machting string [Priority: 1] and loops till I stop syslog-ng!!!
>
> Sensor1:~# /etc/init.d/syslog-ng stop
> Stopping system logging: syslog-ng.
> Sensor1:~# tail -f /tmp/schrott
> AAA
> Do 20. Mär 14:29:12 CET 2008
> AAA
> Do 20. Mär 14:29:12 CET 2008
> AAA
> Do 20. Mär 14:29:12 CET 2008
> AAA
> Do 20. Mär 14:29:12 CET 2008
> AAA
> Do 20. Mär 14:29:12 CET 2008
Here is the explanation:
Your script appends to /tmp/schrott and exits. Syslog-ng restarts it.
Your script appends to /tmp/schrott and exits. Syslog-ng restarts it.
Your script appends to /tmp/schrott and exits. Syslog-ng restarts it.
Your script appends to /tmp/schrott and exits. Syslog-ng restarts it.
Your script appends to /tmp/schrott and exits. Syslog-ng restarts it.
... and so on, forever ...
The reason is that the script is not obeying the rules for a syslog-ng
destination, as previously stated.
-----Original Message-----
From: syslog-ng-bounces at lists.balabit.hu [mailto:syslog-ng-bounces at lists.balabit.hu] On Behalf Of Fegan, Joe
Sent: 20 March 2008 15:11
To: Syslog-ng users' and developers' mailing list
Subject: Re: [syslog-ng] loop caused by syslog-ng filter
Hi Adam,
Please listen to the advice you're being given. From your examples, I think you believe that syslog-ng starts a new instance of your script for each new message. This is not true. Syslog-ng starts *one* instance of your script during its own startup, and it expects that one instance to stay alive for a long time. The script must read lines from its stdin *in a loop* until syslog-ng hangs up the connection (which will happen when syslog-ng is shutting down). Each line is an individual message.
Both of your example scripts exit immediately. You can't do that. Syslog-ng will automatically restart any destination script that exits, so your scripts will be restarted over and over and over and over because they keep exiting. Eventually syslog-ng will go crazy doing this.
As Christian said, a destination script needs a "read" loop like this example:
#!/bin/bash
while
read MSG
do
# Process the message $MSG. Do whatever you like with it.
# Email it to someone. Whatever. Here we put it in a file.
echo $MSG >> /tmp/schrott
# We're finished with this message, wait for the next one.
done
# Read returned eof. This means "no more messages". Exit.
exit 0
Christian's tips for testing destination scripts are also good. Run the script from the command line yourself. Type in messages like the ones you expect syslog-ng to send it (cut/paste them from /var/log/messages or wherever). Check that the script does what you expect with each message and that it waits for the next one, does *not* exit immediately. When it is working interactively then you can hook it up to syslog-ng.
Joe.
-----Original Message-----
From: syslog-ng-bounces at lists.balabit.hu [mailto:syslog-ng-bounces at lists.balabit.hu] On Behalf Of JUNG, Christian
Sent: 20 March 2008 14:22
To: Syslog-ng users' and developers' mailing list
Subject: Re: [syslog-ng] loop caused by syslog-ng filter
Okay. Don't know if I get the things right, but:
All scripts you've mailed to the list write something to a file or send a mail if they're called and terminate directly after that action. syslog-ng will respawn (neu starten) them directly.
The program started by the program destination should not terminate itself. It has to listen on STDIN for a log message given from syslog-ng. Afterwards it can do something useful and then it has to listen for the next message.
In a shell script you can do this with a "while read LINE; do ...; done".
Try this:
---8<---
#!/bin/bash
while read LINE; do
echo $(date) $LINE >> /tmp/schrott
done
---8<---
Execute this script on the command line, enter some random stuff and look into /tmp/schrott:
user at box:~> ./test-script
bla bla bla
bla
Terminate this script by pressing CTRL-D. You should see something like this in /tmp/schrott:
Thu Mar 20 15:13:16 CET 2008 bla bla bla
Thu Mar 20 15:13:18 CET 2008 bla
If not your script doesn't work :-) See for typos.
If this works, put it in your syslog-ng conf. It should do the same (only prio 1 messages should be visible with the date prepended).
If this works well, try this script first on the command line:
---8<---
#!/bin/bash
while read LINE; do
mail -s "High Priority Snort Alert" Sub-Zero at xxx.de <<-EOF
Alert, Priority 1
$LINE
EOF
done
---8<---
You should receive for every given input line exactly one mail.
If this works put it in your syslog-ng.conf. Now you should be done.
The thread you mentioned is about two running scripts where only one should run. This shouldn't be the case here. But you can have a look at the output of "ps fax". If you see multiple processes under syslog-ng then you might have the problem.
bye
Chris
> -----Original Message-----
> From: syslog-ng-bounces at lists.balabit.hu
> [mailto:syslog-ng-bounces at lists.balabit.hu]On Behalf Of Adam Richter
> Sent: Thursday, March 20, 2008 2:46 PM
> To: Syslog-ng users' and developers' mailing list
> Subject: Re: [syslog-ng] loop caused by syslog-ng filter
>
>
> Hi!
>
> Not working! Syslog-ng filters for exactly the string: [Priority: 1]
> and not as it is piped by the mail script: #priority 1# note the ":" !
> Anyway I used your script --> same fault!!!
> I have also used following script:
>
>
> #!/bin/sh
>
> echo AAA >> /tmp/schrott
> date >> /tmp/schrott
>
> There is no output like [Priority: 1]!!!
> Then I did following: tail -f /tmp/schrott and got a loop
> too! The loop starts when syslog-ng recognices the first
> machting string [Priority: 1] and loops till I stop syslog-ng!!!
>
>
> Sensor1:~# /etc/init.d/syslog-ng stop
> Stopping system logging: syslog-ng.
> Sensor1:~# tail -f /tmp/schrott
> AAA
> Do 20. Mär 14:29:12 CET 2008
> AAA
> Do 20. Mär 14:29:12 CET 2008
> AAA
> Do 20. Mär 14:29:12 CET 2008
> AAA
> Do 20. Mär 14:29:12 CET 2008
> AAA
> Do 20. Mär 14:29:12 CET 2008
>
>
> I think it has something in common with this thread:
>
> https://lists.balabit.hu/pipermail/syslog-ng/2006-October/009454.html
>
> Any other ideas? It´s very important!
>
> German: Es handelt sich hier um meine Abschlussprüfung, und
> dies ist der letzte Fehler der Auftritt, ansonsten läuft das Projekt.
>
> bye, Adam / Sub-Zero !
>
>
> -------- Original-Nachricht --------
> > Datum: Thu, 20 Mar 2008 10:32:31 +0100
> > Von: "JUNG, Christian" <christian.jung at saarstahl.com>
> > An: "Syslog-ng users\' and developers\' mailing list"
> <syslog-ng at lists.balabit.hu>
> > Betreff: Re: [syslog-ng] loop caused by syslog-ng filter
>
> > Hi Adam,
> >
> > syslog-ng does the right thing :-).
> >
> > It starts the program/script once and pipes on STDIN every
> log-message
> > which matches the filter.
> >
> > If your script is started, it will call mail and pipe
> "Alert, priority 1"
> > to its STDIN and then exits. syslog-ng sees this and
> restarts it (version
> > 2.0 or higher behave that way, see
> >
> <http://www.balabit.com/dl/html/syslog-ng-admin-guide_en.html/
> ch09s02.html#reference_destination_program>).
> >
> > For you purpose this would be better:
> >
> > ---8<---
> > #!/bin/bash
> >
> > while read LINE; do
> > cat <<-EOF | mail -s "High Priority Snort Alert"
> Sub-Zero at xxx.de
> > Alert, Priority 1
> > $LINE
> > EOF
> > done
> > ---8<---
> >
> >
> > bye
> > Chris
> >
> > >
> > > -----Original Message-----
> > > From: syslog-ng-bounces at lists.balabit.hu
> > > [mailto:syslog-ng-bounces at lists.balabit.hu]On Behalf Of
> Adam Richter
> > > Sent: Thursday, March 20, 2008 9:23 AM
> > > To: syslog-ng at lists.balabit.hu
> > > Subject: [syslog-ng] loop caused by syslog-ng filter
> > >
> > >
> > > Hi!
> > >
> > > First off, sorry for my poor english!
> > > I have a problem with a loop caused by syslog-ng v. 2.0.8. I
> > > have set up Snort as an IDS System. Snort writes its messages
> > > in unified-format to /var/log/snort/snort.alert and
> > > /var/log/snort/snort.log. There are two Barnyard processes
> > > which read the unified files and convert it to messages that
> > > syslog and MySQL understand. Syslog-ng writes the messages
> > > to /var/log/auth.log. All this is working fine. Now, I want
> > > to set up a filter for Priority 1 alerts. This alert should
> > > be send to the Administrator.
> > >
> > > I used following filter for syslog-ng:
> > >
> > > source src {unix-stream("/dev/log"); internal();};
> > > destination email{program("/usr/local/bin/alert_mail.sh");};
> > > filter high {match("[Priority: 1]");};
> > > log {source(src);filter(high); destination(email);};
> > >
> > >
> > > The alert_mail.sh:
> > >
> > > #!/bin/sh
> > > cat << EOF | mail -s "High Priority Snort Alert" Sub-Zero at xxx.de
> > > Alert, Priority 1
> > > EOF
> > >
> > >
> > > Then I use Nessus to cause some alerts with Priority 1. I can
> > > see 4 alerts with the Priority 1 with BASE and in
> /var/log/auth.log.
> > >
> > > Syslog-ng recognises the alert with Priority 1 and activates
> > > the script /usr/local/bin/alert_mail.sh
> > >
> > > All this is working, but the script is restarted by syslog-ng
> > > again an again.
> > >
> > > Extract from /var/log/messages:
> > >
> > > Mar 18 15:46:16 Sensor1 syslog-ng[5191]: Child program
> > > exited, restarting; cmdline='/usr/local/bin/alert_mail.sh ',
> > > status='0'
> > > Mar 18 15:46:16 Sensor1 syslog-ng[5191]: Starting destination
> > > program; cmdline='/usr/local/bin/alert_mail.sh '
> > > Mar 18 15:46:16 Sensor1 syslog-ng[5191]: Closing log writer
> > > fd; fd='11'
> > > Mar 18 15:46:16 Sensor1 syslog-ng[5191]: Child program
> > > exited, restarting; cmdline='/usr/local/bin/alert_mail.sh ',
> > > status='0'
> > > Mar 18 15:46:16 Sensor1 syslog-ng[5191]: Starting destination
> > > program; cmdline='/usr/local/bin/alert_mail.sh '
> > > Mar 18 15:46:16 Sensor1 syslog-ng[5191]: Child program
> > > exited, restarting; cmdline='/usr/local/bin/alert_mail.sh ',
> > > status='0'
> > > Mar 18 15:46:16 Sensor1 syslog-ng[5191]: Starting destination
> > > program; cmdline='/usr/local/bin/alert_mail.sh '
> > > ...
> > > Mar 18 15:42:00 Sensor1 syslog-ng[17354]: Child program
> > > exited, restarting; cmdline='/usr/local/bin/alert_mail.sh ',
> > > status='256'
> > > Mar 18 15:42:00 Sensor1 syslog-ng[17354]: Starting
> > > destination program; cmdline='/usr/local/bin/alert_mail.sh '
> > > Mar 18 15:42:00 Sensor1 syslog-ng[17354]: Child program
> > > exited, restarting; cmdline='/usr/local/bin/alert_mail.sh ',
> > > status='256'
> > > Mar 18 15:42:00 Sensor1 syslog-ng[17354]: Starting
> > > destination program; cmdline='/usr/local/bin/alert_mail.sh '
> > > Mar 18 15:42:00 Sensor1 syslog-ng[17354]: Child program
> > > exited, restarting;
> > >
> > > ...
> > >
> > >
> > > I get thousands of mails per minute till I stop syslog-ng.
> > >
> > > Output of /var/log/auth.log(so y see that syslog-ng writes
> > > snort/barnyard messages correctly to auth.log):
> > >
> > > Mar 19 13:56:54 src at Sensor1 barnyard: [1:1394:8] SHELLCODE
> > > x86 NOOP [Classification: Executable code was detected]
> > > [Priority: 1] {UDP} 172.25.1.152:4758 -> 172.28.100.10:137
> > > Mar 19 13:57:13 src at Sensor1 barnyard: [1:1446:8] SMTP vrfy
> > > root [Classification: Attempted Information Leak] [Priority:
> > > 2] {TCP} 172.25.1.152:1085 -> 172.28.100.10:25
> > > Mar 19 13:57:13 src at Sensor1 barnyard: [1:660:11] SMTP expn
> > > root [Classification: Attempted Information Leak] [Priority:
> > > 2] {TCP} 172.25.1.152:1085 -> 172.28.100.10:25
> > > Mar 19 13:57:22 src at Sensor1 barnyard: [1:12626:2] Snort Alert
> > > [1:12626:0] [Classification: Decode of an RPC Query]
> > > [Priority: 2] {UDP} 172.25.1.152:1146 -> 172.28.100.10:111
> > > Mar 19 13:57:22 src at Sensor1 barnyard: [1:585:9] RPC portmap
> > > sadmind request UDP [Classification: Decode of an RPC Query]
> > > [Priority: 2] {UDP} 172.25.1.152:1146 -> 172.28.100.10:111
> > > Mar 19 13:57:24 src at Sensor1 barnyard: [1:566:6] POLICY
> > > PCAnywhere server response [Classification: Misc activity]
> > > [Priority: 3] {UDP} 172.25.1.152:1155 -> 172.28.100.10:5632
> > > Mar 19 15:11:27 src at Sensor1 barnyard: [122:1:0] portscan: TCP
> > > Portscan [Classification: Unknown] [Priority: 3] {PROTO255}
> > > 172.25.1.152 -> 172.28.100.10
> > > Mar 19 15:11:58 src at Sensor1 barnyard: [1:1420:13] SNMP trap
> > > tcp [Classification: Attempted Information Leak] [Priority:
> > > 2] {TCP} 172.25.1.152:4482 -> 172.28.100.10:162
> > > Mar 19 15:11:58 src at Sensor1 barnyard: [1:1418:13] SNMP
> > > request tcp [Classification: Attempted Information Leak]
> > > [Priority: 2] {TCP} 172.25.1.152:4482 -> 172.28.100.10:161
> > > Mar 19 15:12:05 src at Sensor1 barnyard: [1:1421:13] SNMP
> > > AgentX/tcp request [Classification: Attempted Information
> > > Leak] [Priority: 2] {TCP} 172.25.1.152:4482 -> 172.28.100.10:705
> > > Mar 19 15:12:16 src at Sensor1 barnyard: [122:1:0] portscan: TCP
> > > Portscan [Classification: Unknown] [Priority: 3] {PROTO255}
> > > 172.25.1.152 -> 172.28.100.10
> > > Mar 19 15:12:19 src at Sensor1 barnyard: [1:1394:8] SHELLCODE
> > > x86 NOOP [Classification: Executable code was detected]
> > > [Priority: 1] {UDP} 172.25.1.152:137 -> 172.28.100.10:137
> > > Mar 19 15:12:20 src at Sensor1 barnyard: [1:1394:8] SHELLCODE
> > > x86 NOOP [Classification: Executable code was detected]
> > > [Priority: 1] {UDP} 172.25.1.152:137 -> 172.28.100.10:137
> > > Mar 19 15:12:22 src at Sensor1 barnyard: [1:1394:8] SHELLCODE
> > > x86 NOOP [Classification: Executable code was detected]
> > > [Priority: 1] {UDP} 172.25.1.152:137 -> 172.28.100.10:137
> > >
> > >
> > > I think it has something in common with this topic:
> > >
> > >
https://lists.balabit.hu/pipermail/syslog-ng/2006-October/009454.html
> >
> > Thanks in advance!
> >
> > Sub-Zero
> >
> >
> > --
> > GMX startet ShortView.de. Hier findest Du Leute mit Deinen Interessen!
> > Jetzt dabei sein: http://www.shortview.de/?mc=sv_ext_mf@gmx
> > ______________________________________________________________
> > ________________
> > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
--
Psst! Geheimtipp: Online Games kostenlos spielen bei den GMX Free Games!
http://games.entertainment.gmx.net/de/entertainment/games/free
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.campin.net/syslog-ng/faq.html
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.campin.net/syslog-ng/faq.html
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.campin.net/syslog-ng/faq.html
More information about the syslog-ng
mailing list