[syslog-ng] Please help getting patterndb working.

[Nick]

Hello,

By following the documentation examples I think I have recreated a
custom parser, but it doesn't work, please can you advise what I have
missed?

To create my patterndb I have used the example in link [1], to load it
and output to a file I have kinda copied the example in link [2].

The issue I have is that my output file is mostly empty, it's like the
variables are not being read properly.

In case the files do not render properly , please see this gist ->
https://gist.github.com/linickx/8002981

I have built a clean fedora 19 box which ships with syslog-ng-3.4.6-1.

My /etc/syslog-ng/syslog-ng conf looks like:

<PASTE>
@version:3.2

# Default options copied from distribution
options {

        flush_lines (0);
        time_reopen (10);
        log_fifo_size (1000);
        long_hostnames (off);
        use_dns (no);
        use_fqdn (no);
        create_dirs (yes);
        keep_hostname (yes);
        owner("nick");
        group("nick");
        perm(0640);
};

source test_logfile {
        file("/home/nick/testfile.log");
};

parser test_pattern {
            db_parser(
                file("/home/nick/example.xml")
            );
        };

destination test_output {
        file("/home/nick/output.log"
                template("${SSH_USERNAME}; ${SSH_CLIENT_ADDRESS}; \n")
                template_escape(no)
        );
};

log { source (test_logfile); parser(test_pattern); destination (test_output); };
</PASTE>

The /home/nick/example.xml looks like

<PASTE>
<patterndb version='4' pub_date='2010-10-17'>
    <ruleset name='ssh' id='123456678'>
        <pattern>ssh</pattern>
            <rules>
                <rule provider='me' id='182437592347598' class='system'>
                    <patterns>
                        <pattern>Accepted @QSTRING:SSH.AUTH_METHOD: @
for at QSTRING:SSH_USERNAME: @from\ @QSTRING:SSH_CLIENT_ADDRESS: @port
@NUMBER:SSH_PORT_NUMBER:@ ssh2</pattern>
                    </patterns>
                    <examples>
                        <example>
                            <test_message>Accepted password for
sampleuser from 10.50.0.247 port 42156 ssh2</test_message>
                            <test_values>
                                <test_value
name="SSH.AUTH_METHOD">password</test_value>
                                <test_value
name="SSH_USERNAME">sampleuser</test_value>
                                <test_value
name="SSH_CLIENT_ADDRESS">10.50.0.247</test_value>
                                <test_value
name="SSH_PORT_NUMBER">42156</test_value>
                            </test_values>
                       </example>
                    </examples>
                </rule>
            </rules>
    </ruleset>
</patterndb>
</PASTE>

For /home/nick/testfile.log I have used

<PASTE>
Accepted password for sampleuser from 10.50.0.247 port 42156 ssh2
Accepted password for user from 10.51.0.27 port 4256 ssh2
</PASTE>

after starting syslog-ng I get the following in /home/nick/output.log

PASTE
; ;
; ;
</PASTE>

Any pointers on what I'm doing wrong would be very much appreciated.

This is a cross-post from
http://communities.balabit.com/balabit/topics/please_help_me_get_patterndb_working,
apologies if this offends, the balabit.com website is not clear on
where the most active community can be found.

Thanks in Advance,
Nick


[1] http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-3.3-guides/en/syslog-ng-ose-v3.3-guide-admin-en/html/reference_patterndb_schemes.html#patterndb-schema-context-scope

[2] http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-3.4-guides/en/syslog-ng-ose-v3.4-guide-admin/html/configuring-pattern-databases.html