[syslog-ng] pattern matching on xxx#

[Martin Holste]

I'll chime in here to once again recommending piping to Perl using
program() if you have crazy stuff to do.  In your case, you could have
a very simple (one liner, really) script that does the regex hostname
rewrite so that hostXX would get rewritten to just XX or something
easy for syslog-ng to filter on and route to the appropriate
destination.  Just have a socket source available as the destination
from Perl and a source in syslog-ng to complete the circuit.

On Fri, Oct 15, 2010 at 2:13 PM, Bill Anderson
<Bill.Anderson at bodybuilding.com> wrote:
>
> On Oct 15, 2010, at 1:01 PM, Matthew Hall wrote:
>
>> On Fri, Oct 15, 2010 at 12:48:14PM -0600, Bill Anderson wrote:
>>> I have hostnames of the format xxxx# such as host1, hostb1, hostc1. I
>>> need to split that into two fields such as (host,1).
>>>
>>> Unfortunately, since @@ escapes the @ and STRING and it's followers
>>> ALSO match digits, I've not found the obvious means to get that out.
>>> Conceptually something like @LETTER:host.name@@NUMBER:host.id@ woudl
>>> do it, save that LETTER doesn't exist and @@ escapes.
>>
>> I think you can get around @@ escapes by adding extra @'s. ;-) Too bad
>> LETTER doesn't exist yet. Man I wish it did for some of the @#$%^&* @#$%
>> I have to parse.
>
> Hmm if @@@ worked, and LETTER existed, that *would* solve it.
>
>>
>>> The end goal is as follows (pseudo-code):
>>> I need to have a destination for each (HOST). For example all files
>>> from hosta## go to /var/log/hosta/ and entries for hostb## go to
>>> /var/log/hostb/
>>
>> Goal makes sense for a big server farm. Crazy idea. Depending how your
>> IP subnets are set up... could you break the host IPs into pieces using
>> '.' and direct the logs where they need to go using the IP?
>
> Hmm an interesting idea. Not sure, but will look into it.
>
>>
>> Can you try the rewrite on a second syslog-ng receiving a relayed copy
>> of the traffic using AF_UNIX SOCK_DGRAM also known as unix-dgram driver?
>> That way if it has disastrous side effects you could find out without
>> causing outages in your primary syslog-ng. I often use this sort of
>> approach for testing crazy ideas.
>
> Yeah I've abused the daylights out of some of my syslog-ng installs using things like this. Even to the point of having a destination be a network socket that did some conversion to binary that I then shipped back into SNG which then wrote that to files. Saved me from writing the code to manage the files (and let me store them on a different server). ;) Hmm, perhaps the rewrite would be performance-safe if done by the SNG clients as opposed to the Central Log Servers (CLS).
>
>>
>>> I'm open to suggestions (that don't involve changing server names,
>>> preferably ;) ) as to how to accomplish this.
>>
>> Let's keep working on it until we come up with a good idea. There has to
>> be some way to make it happen.
>
>
> I'm sure there is a way, rest assured it *will* be found. ;)
>
> Now, if I could set variables in the conf file to be used in templates and filters ... ;)
>
>
> Cheers,
> Bill
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
>