[syslog-ng] pattern matching on xxx#

[Bill Anderson]

On Oct 15, 2010, at 1:01 PM, Matthew Hall wrote:

> On Fri, Oct 15, 2010 at 12:48:14PM -0600, Bill Anderson wrote:
>> I have hostnames of the format xxxx# such as host1, hostb1, hostc1. I 
>> need to split that into two fields such as (host,1).
>> 
>> Unfortunately, since @@ escapes the @ and STRING and it's followers 
>> ALSO match digits, I've not found the obvious means to get that out. 
>> Conceptually something like @LETTER:host.name@@NUMBER:host.id@ woudl 
>> do it, save that LETTER doesn't exist and @@ escapes.
> 
> I think you can get around @@ escapes by adding extra @'s. ;-) Too bad 
> LETTER doesn't exist yet. Man I wish it did for some of the @#$%^&* @#$% 
> I have to parse.

Hmm if @@@ worked, and LETTER existed, that *would* solve it.

> 
>> The end goal is as follows (pseudo-code):
>> I need to have a destination for each (HOST). For example all files 
>> from hosta## go to /var/log/hosta/ and entries for hostb## go to 
>> /var/log/hostb/
> 
> Goal makes sense for a big server farm. Crazy idea. Depending how your 
> IP subnets are set up... could you break the host IPs into pieces using 
> '.' and direct the logs where they need to go using the IP?

Hmm an interesting idea. Not sure, but will look into it.

> 
> Can you try the rewrite on a second syslog-ng receiving a relayed copy 
> of the traffic using AF_UNIX SOCK_DGRAM also known as unix-dgram driver? 
> That way if it has disastrous side effects you could find out without 
> causing outages in your primary syslog-ng. I often use this sort of 
> approach for testing crazy ideas.

Yeah I've abused the daylights out of some of my syslog-ng installs using things like this. Even to the point of having a destination be a network socket that did some conversion to binary that I then shipped back into SNG which then wrote that to files. Saved me from writing the code to manage the files (and let me store them on a different server). ;) Hmm, perhaps the rewrite would be performance-safe if done by the SNG clients as opposed to the Central Log Servers (CLS).

> 
>> I'm open to suggestions (that don't involve changing server names, 
>> preferably ;) ) as to how to accomplish this.
> 
> Let's keep working on it until we come up with a good idea. There has to 
> be some way to make it happen.


I'm sure there is a way, rest assured it *will* be found. ;)

Now, if I could set variables in the conf file to be used in templates and filters ... ;)


Cheers,
Bill