OK folks, this has come up (again). Seems that the ArcSight parser is not intelligent enough to handle messages coming from syslog-ng after being forwarded along. So I need some advice on how to handle this issue. First, some background... I added our ArcSight server as a syslog-ng target some time ago. The folks who use the ArcSight stuff emailed me and said that the parser for ArcSight could not handle parsing the messages coming from syslog-ng, because of the prepending of the server time to the syslog-ng message. Here is an excerpt from one of the emails from their support folks: In looking through all the information, I see that there are lot of parsing issues, all due to what look like malformed syslog messages. For example: Apr 25 22:54:24 x.x.x.x router/router 99228: Apr 25 22:54:23.474 UTC: %CDP-4-DUPLEX_MISMATCH: duplex mismatch discovered on GigabitEthernet1/0/1 (not full duplex), with switch FastEthernet0/1 (full duplex). This is a raw event from the export. Notice that the second timestamp forward is the actual message, which is CDP, so from a cisco switch or some layer 2 device. The actual event from the cisco device should look like as follows, which is what our parser is designed to work with: Apr 25 22:54:23.474 UTC: %CDP-4-DUPLEX_MISMATCH: duplex mismatch discovered on GigabitEthernet1/0/1 (not full duplex), with switch FastEthernet0/1 (full duplex). Another excerpt... I looked over the information you had uploaded already, and is actually a common issue. When syslog events are forwarded from one syslog server to another syslog server, or pipe, or file, the forwarding syslog server prepends timestamp and other information, which makes the message unusable. We require syslog message to adhere to the standard RFC syslog format for the connector to read them, and when forwarding syslog messages that is not the case and we are unable to support that configuration. So, the question is what to do about it. I apparently need to send this information on to the ArcSight server without the prepended data (the "Apr 25 22:54:24 x.x.x.x router/router 99228:" portion of the message from the first email excerpt), but I need to keep it in place for EVERY other target I am sending to. Can anyone tell me what are my options here, please? Thanks a LOT in advance!!! (Bazsi, please feel free to chime in on this one! LOL) Chris Ivey Affiliated Computer Services Enterprise Management Integration Services Infrastructure Management Senior Analyst chris.ivey@acs-inc.com "I have not failed, I have simply found 10,000 ways which do not work!" -- Thomas Edison "When you find yourself in a hole, the best thing to do is stop digging!" -- Nick Stokes "I reject your reality, and substitute my own!" -- Adam Savage
Actually, the cisco messages don't conform to the RFC, that's why syslog-ng puts in a proper header section. You should ask them to read the RFC (which was an observational undertaking, seen as a pre-req for the next-gen syslog RFC work). They're quite wrong. ...... Original Message ....... On Fri, 14 Sep 2007 15:49:02 -0500 "Ivey, Chris" <Chris.ivey@acs-inc.com> wrote:
OK folks, this has come up (again). Seems that the ArcSight parser is not intelligent enough to handle messages coming from syslog-ng after being forwarded along. So I need some advice on how to handle this issue. First, some background...
I added our ArcSight server as a syslog-ng target some time ago. The folks who use the ArcSight stuff emailed me and said that the parser for ArcSight could not handle parsing the messages coming from syslog-ng, because of the prepending of the server time to the syslog-ng message. Here is an excerpt from one of the emails from their support folks:
In looking through all the information, I see that there are lot of parsing issues, all due to what look like malformed syslog messages.
For example: Apr 25 22:54:24 x.x.x.x router/router 99228: Apr 25 22:54:23.474 UTC: %CDP-4-DUPLEX_MISMATCH: duplex mismatch discovered on GigabitEthernet1/0/1 (not full duplex), with switch FastEthernet0/1 (full duplex).
This is a raw event from the export. Notice that the second timestamp forward is the actual message, which is CDP, so from a cisco switch or some layer 2 device.
The actual event from the cisco device should look like as follows, which is what our parser is designed to work with: Apr 25 22:54:23.474 UTC: %CDP-4-DUPLEX_MISMATCH: duplex mismatch discovered on GigabitEthernet1/0/1 (not full duplex), with switch FastEthernet0/1 (full duplex).
Another excerpt...
I looked over the information you had uploaded already, and is actually a common issue. When syslog events are forwarded from one syslog server to another syslog server, or pipe, or file, the forwarding syslog server prepends timestamp and other information, which makes the message unusable.
We require syslog message to adhere to the standard RFC syslog format for the connector to read them, and when forwarding syslog messages that is not the case and we are unable to support that configuration.
So, the question is what to do about it. I apparently need to send this information on to the ArcSight server without the prepended data (the "Apr 25 22:54:24 x.x.x.x router/router 99228:" portion of the message from the first email excerpt), but I need to keep it in place for EVERY other target I am sending to. Can anyone tell me what are my options here, please? Thanks a LOT in advance!!!
(Bazsi, please feel free to chime in on this one! LOL)
Chris Ivey
Affiliated Computer Services Enterprise Management Integration Services Infrastructure Management Senior Analyst
chris.ivey@acs-inc.com
There is a reason they don't comply it makes some of their older monitoring tools work better(Pre SNMP). one thing I noticed is they seem to put a sequence number in the header of the message. I doubt that cisco will fix the headers because its simply not in their best intrest to do so On Fri, 14 Sep 2007 11:02 pm, Concatenate wrote:
Actually, the cisco messages don't conform to the RFC, that's why syslog-ng puts in a proper header section.
You should ask them to read the RFC (which was an observational undertaking, seen as a pre-req for the next-gen syslog RFC work). They're quite wrong.
...... Original Message ....... On Fri, 14 Sep 2007 15:49:02 -0500 "Ivey, Chris" <Chris.ivey@acs-inc.com> wrote:
OK folks, this has come up (again). Seems that the ArcSight parser is not intelligent enough to handle messages coming from syslog-ng after being forwarded along. So I need some advice on how to handle this issue. First, some background...
I added our ArcSight server as a syslog-ng target some time ago. The folks who use the ArcSight stuff emailed me and said that the parser for ArcSight could not handle parsing the messages coming from syslog-ng, because of the prepending of the server time to the syslog-ng message. Here is an excerpt from one of the emails from their support folks:
In looking through all the information, I see that there are lot of parsing issues, all due to what look like malformed syslog messages.
For example: Apr 25 22:54:24 x.x.x.x router/router 99228: Apr 25 22:54:23.474 UTC: %CDP-4-DUPLEX_MISMATCH: duplex mismatch discovered on GigabitEthernet1/0/1 (not full duplex), with switch FastEthernet0/1 (full duplex).
This is a raw event from the export. Notice that the second timestamp forward is the actual message, which is CDP, so from a cisco switch or some layer 2 device.
The actual event from the cisco device should look like as follows, which is what our parser is designed to work with: Apr 25 22:54:23.474 UTC: %CDP-4-DUPLEX_MISMATCH: duplex mismatch discovered on GigabitEthernet1/0/1 (not full duplex), with switch FastEthernet0/1 (full duplex).
Another excerpt...
I looked over the information you had uploaded already, and is actually a common issue. When syslog events are forwarded from one syslog server to another syslog server, or pipe, or file, the forwarding syslog server prepends timestamp and other information, which makes the message unusable.
We require syslog message to adhere to the standard RFC syslog format for the connector to read them, and when forwarding syslog messages that is not the case and we are unable to support that configuration.
So, the question is what to do about it. I apparently need to send this information on to the ArcSight server without the prepended data (the "Apr 25 22:54:24 x.x.x.x router/router 99228:" portion of the message from the first email excerpt), but I need to keep it in place for EVERY other target I am sending to. Can anyone tell me what are my options here, please? Thanks a LOT in advance!!!
(Bazsi, please feel free to chime in on this one! LOL)
Chris Ivey
Affiliated Computer Services Enterprise Management Integration Services Infrastructure Management Senior Analyst
chris.ivey@acs-inc.com
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
The average person does a lot of work in the name of laziness! Save youre self the effort by doing it right the first time. Do it with free speech software.
OBTW, use a template for that destination. IIRC the $MSG macro sent over to them will be exactly what you need. Just change the destination to a file for testing and see what your template does, easy to test that way. When it's all good change it back. ...... Original Message ....... On Fri, 14 Sep 2007 15:49:02 -0500 "Ivey, Chris" <Chris.ivey@acs-inc.com> wrote:
OK folks, this has come up (again). Seems that the ArcSight parser is not
intelligent enough to handle messages coming from syslog-ng after being forwarded along. So I need some advice on how to handle this issue. First, some background...
I added our ArcSight server as a syslog-ng target some time ago. The
folks who use the ArcSight stuff emailed me and said that the parser for ArcSight could not handle parsing the messages coming from syslog-ng, because of the prepending of the server time to the syslog-ng message. Here is an excerpt from one of the emails from their support folks:
In looking through all the information, I see that there are lot of
parsing issues, all due to what look like malformed syslog messages.
For example: Apr 25 22:54:24 x.x.x.x router/router 99228: Apr 25 22:54:23.474 UTC:
%CDP-4-DUPLEX_MISMATCH: duplex mismatch discovered on GigabitEthernet1/0/1 (not full duplex), with switch FastEthernet0/1 (full duplex).
This is a raw event from the export. Notice that the second timestamp
forward is the actual message, which is CDP, so from a cisco switch or some layer 2 device.
The actual event from the cisco device should look like as follows, which
is what our parser is designed to work with:
Apr 25 22:54:23.474 UTC: %CDP-4-DUPLEX_MISMATCH: duplex mismatch discovered on GigabitEthernet1/0/1 (not full duplex), with switch FastEthernet0/1 (full duplex).
Another excerpt...
I looked over the information you had uploaded already, and is actually a common issue. When syslog events are forwarded from one syslog server to another syslog server, or pipe, or file, the forwarding syslog server prepends timestamp and other information, which makes the message unusable.
We require syslog message to adhere to the standard RFC syslog format for the connector to read them, and when forwarding syslog messages that is not the case and we are unable to support that configuration.
So, the question is what to do about it. I apparently need to send this information on to the ArcSight server without the prepended data (the "Apr 25 22:54:24 x.x.x.x router/router 99228:" portion of the message from the first email excerpt), but I need to keep it in place for EVERY other target I am sending to. Can anyone tell me what are my options here, please? Thanks a LOT in advance!!!
(Bazsi, please feel free to chime in on this one! LOL)
Chris Ivey
Affiliated Computer Services Enterprise Management Integration Services Infrastructure Management Senior Analyst
chris.ivey@acs-inc.com
"I have not failed, I have simply found 10,000 ways which do not work!" -- Thomas Edison "When you find yourself in a hole, the best thing to do is stop digging!" -- Nick Stokes "I reject your reality, and substitute my own!" -- Adam Savage
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
participants (3)
-
Concatenate
-
Ivey, Chris
-
Paul Robert Marino