[syslog-ng] Problem using tags with syslog-ng 3.1

Zoltán Pallagi pzolee at balabit.hu
Fri Apr 2 09:38:27 CEST 2010


Martin Holste wrote:
> Ok, I think I see what you're saying: the tag only exists on the local 
> box and does not get forwarded in the message.  You were saying you 
> have to overwrite the original program with some other value so that 
> the tag is permanent and will survive multiple relays.  Sorry for the 
> confusion. 
>
> So the difference is that tags are only supported on some types of 
> sources?
Please, read our guide about tagging and you will find the answers of 
your questions: 
http://www.balabit.hu/dl/html/syslog-ng-ose-v3.1-guide-admin-en.html/ch04s06.html#tagging_messages

The only missing part is that the sent(or forwarded) message will not 
contain these tags (so it's not a macro and you cannot use them in a 
template string). I will ask our tech writer to include it into the 
documentation.
>
> On Thu, Apr 1, 2010 at 9:40 AM, Zoltán Pallagi <pzolee at balabit.hu 
> <mailto:pzolee at balabit.hu>> wrote:
>
>     Martin Holste wrote:
>>     Ok, so '.sources' has nothing to do with user-provided tags, but
>>     in his example Marci uses:
>>
>>     source s_tcp2 {
>>     tcp(ip(192.168.1.2) port(1514) tags("tcp", "windows));
>>     };
>>
>>     #Match on tags "tcp" or "udp"
>>     filter f_net {
>>     tags("tcp", "udp");
>>     };
>>
>>     Which seems to imply that arbitrary user tags can be set and then
>>     matched on in a filter later.  So, it seems what is missing from
>>     Stefan's config was:
>>
>>     source s_remote { tcp (ip("0.0.0.0") port(13074) keep-alive(yes)
>>     tags("log2"); };
>>
>>     Which would allow his later filter statement
>>
>>     filter f_log2 { host("web00(09|10)") and tags("log2"); };
>>
>>     to succeed.  Right?
>     Yes and no. It's a correct way to use tagging on sources.
>
>     And no, because his original aim wasn't that.
>     He wanted to mark logs coming from different source files of the
>     client by tags and than to recreate almost the same file and log
>     structure on the server by these tags.
>
>     In your solution he would have only one tag for all the messages
>     coming from different sources of client.
>
>>
>>     On Thu, Apr 1, 2010 at 8:37 AM, Zoltán Pallagi <pzolee at balabit.hu
>>     <mailto:pzolee at balabit.hu>> wrote:
>>
>>         Martin Holste wrote:
>>>         Please step in and correct me if I'm wrong here, but
>>>         according to Marci's blog post at
>>>         http://marci.blogs.balabit.com/2009/05/tag-support-in-syslog-ng.html
>>>         it would appear that this is possible using different
>>>         syntax.  Namely, using tags(".source.log2") in your filter.
>>         No, you are using a wrong tag name.
>>         In this case, you can use the following tag:
>>         tags(".source.s_app")
>>
>>         this is an on-the-fly generated tag,every incoming message
>>         has it one given with the following formula:
>>         ".source.<sourcename>"
>>>
>>>         On Wed, Mar 31, 2010 at 12:57 PM, Zoltán Pallagi
>>>         <pzolee at balabit.hu <mailto:pzolee at balabit.hu>> wrote:
>>>
>>>             Hi,
>>>
>>>             I'm afraid that you may misunderstand the working of
>>>             this feature. The tag field exists only within a running
>>>             syslog-ng and just a virtual part of the message. The
>>>             sent message doesn't contain tag fields that's why you
>>>             cannot filter these tags with another syslog-ng.
>>>
>>>             However, I can suggest you an other solution:
>>>             use the program_override option. This will override the
>>>             $PROGRAM macro with the specified value.
>>>             For example:
>>>             source s_app {
>>>             file("/var/log/log1.log"
>>>             program_override("/var/log/log1.log"));
>>>             file("/opt//log/log2.log" tags("log2")
>>>             program_override("/opt/log/log2.log"));
>>>             file("/opt/log/log3.log" tags("log3")
>>>             program_override("/opt/log/log3.log"));
>>>             };
>>>
>>>             After that, you can use a specified program filter on
>>>             the central logging server side to separate them.
>>>
>>>             2010.03.31. 16:39 keltezéssel, Hoenig, Stefan, VF-Group
>>>             írta:
>>>>             Hi all,
>>>>             I got a problem to get the "tags" feature working on
>>>>             our syslog-ng 3.1. I want to collect messages from 3
>>>>             different files on the
>>>>             source system and want to separate them again on the
>>>>             central logging server.
>>>>             The client configuration looks like this:
>>>>             ----------------------------------------------------------------------------------------------------
>>>>             source s_app {
>>>>             file("/var/log/log1.log");
>>>>             file("/opt//log/log2.log" tags("log2"));
>>>>             file("/opt/log/log3.log" tags("log3"));
>>>>             };
>>>>             options {
>>>>             };
>>>>
>>>>             destination d_app { tcp("logrelay01.domain.com
>>>>             <http://logrelay01.domain.com>" port(13074)); };
>>>>
>>>>             log {
>>>>             source(s_app);
>>>>             destination(d_app);
>>>>             };
>>>>             ----------------------------------------------------------------------------------------------------
>>>>              
>>>>             The log relay does nothing than forward the messages to
>>>>             the central logging server with the following config:
>>>>             ----------------------------------------------------------------------------------------------------
>>>>             options {
>>>>             time_sleep(20);
>>>>             log_fifo_size(1000);
>>>>             dns_cache(2000);
>>>>             dns_cache_expire(87600);
>>>>             keep_hostname(yes);
>>>>             };
>>>>
>>>>             source s_remote { tcp(ip("0.0.0.0") port(13074)); };
>>>>              
>>>>             destination remote_tcp { tcp("centrallog01.domain.com
>>>>             <http://centrallog01.domain.com>" port(13074)); };
>>>>              
>>>>             log {
>>>>             source(s_remote);
>>>>             destination(remote_tcp);
>>>>             };
>>>>             ----------------------------------------------------------------------------------------------------
>>>>              
>>>>             On the central logging server I use filters to separate
>>>>             the logfiles again:
>>>>             ----------------------------------------------------------------------------------------------------
>>>>             @version: 3.0
>>>>             include "/opt/config/syslogng-inc.conf";
>>>>              
>>>>             options {
>>>>             time_sleep(20);
>>>>             dns_cache(2000);
>>>>             dns_cache_expire(87600);
>>>>             keep_hostname(yes);
>>>>             create_dirs(yes);
>>>>             };
>>>>
>>>>             source s_remote { tcp (ip("0.0.0.0") port(13074)
>>>>             keep-alive(yes)); };
>>>>             ============================================
>>>>              
>>>>             This is the confoguration in /opt/config/syslogng-inc.conf
>>>>             # Filter
>>>>             filter f_log1 { host("web00(09|10)"); };
>>>>             filter f_log2 { host("web00(09|10)") and tags("log2"); };
>>>>             filter f_log3 { host("web00(09|10)") and tags("log3"); };
>>>>              
>>>>             #Configuration for Destinations
>>>>             destination d_log1 { file("/var/logs/log1/combined.log"
>>>>             perm(0755) dir_perm(0755)); };
>>>>             destination d_log2 { file("/var/logs/log2/combined.log"
>>>>             perm(0755) dir_perm(0755)); };
>>>>             destination d_log3 { file("/var/logs/log3/combined.log"
>>>>             perm(0755) dir_perm(0755)); };
>>>>             # Logfile log1
>>>>             log {
>>>>             source(s_remote);
>>>>             filter(f_log1);
>>>>             destination(d_log1);
>>>>             };
>>>>              
>>>>             # Logfile log2
>>>>             log {
>>>>             source(s_remote);
>>>>             filter(f_log2);
>>>>             destination(d_log2);
>>>>             };
>>>>              
>>>>             # Logfile log3
>>>>             log {
>>>>             source(s_remote);
>>>>             filter(f_log3);
>>>>             destination(d_log3);
>>>>             };
>>>>             ----------------------------------------------------------------------------------------------------
>>>>              
>>>>             Does anybody have an idea, why it does not work as
>>>>             expected.
>>>>              
>>>>             Thanks for any suggestion and/or idea.
>>>>              
>>>>             Best regards Stefan
>>>>              
>>>>
>>>>
>>>>             ______________________________________________________________________________
>>>>             Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>             Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>>>>             FAQ: http://www.campin.net/syslog-ng/faq.html
>>>>
>>>>               
>>>
>>>
>>>             -- 
>>>             pzolee
>>>
>>>             ______________________________________________________________________________
>>>             Member info:
>>>             https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>             Documentation:
>>>             http://www.balabit.com/support/documentation/?product=syslog-ng
>>>             FAQ: http://www.campin.net/syslog-ng/faq.html
>>>
>>>
>>>
>>>         ------------------------------------------------------------------------
>>>         ______________________________________________________________________________
>>>         Member info:
>>>         https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>         Documentation:
>>>         http://www.balabit.com/support/documentation/?product=syslog-ng
>>>         FAQ: http://www.campin.net/syslog-ng/faq.html
>>
>>
>>         -- 
>>         pzolee
>>             
>>
>>
>>     ------------------------------------------------------------------------
>>     ______________________________________________________________________________
>>     Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>     Documentation:
>>     http://www.balabit.com/support/documentation/?product=syslog-ng
>>     FAQ: http://www.campin.net/syslog-ng/faq.html
>
>
>     -- 
>     pzolee
>         
>
>
> ------------------------------------------------------------------------
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
>   


-- 
pzolee

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20100402/64aaab41/attachment-0001.htm 


More information about the syslog-ng mailing list