[syslog-ng] Please help getting patterndb working.

[Nick]

Hello Fabien,

Thanks for your mail, I am n00b and just following that which balabit
documented, please bare with me :)

[1] I have added a program attribute.

<PASTE>
<patterndb version='4' pub_date='2010-10-17'>
    <ruleset name='ssh' id='123456678'>
        <pattern>ssh</pattern>
            <rules>
                <rule provider='me' id='182437592347598' class='system'>
                    <patterns>
                        <pattern>Accepted @QSTRING:SSH.AUTH_METHOD: @
for at QSTRING:SSH_USERNAME: @from\ @QSTRING:SSH_CLIENT_ADDRESS: @port
@NUMBER
:SSH_PORT_NUMBER:@ ssh2</pattern>
                    </patterns>
                    <examples>
                        <example>
                            <test_message program="example">Accepted
password for sampleuser from 10.50.0.247 port 42156
ssh2</test_message>
                            <test_values>
                                <test_value
name="SSH.AUTH_METHOD">password</test_value>
                                <test_value
name="SSH_USERNAME">sampleuser</test_value>
                                <test_value
name="SSH_CLIENT_ADDRESS">10.50.0.247</test_value>
                                <test_value
name="SSH_PORT_NUMBER">42156</test_value>
                            </test_values>
                       </example>
                    </examples>
                </rule>
            </rules>
    </ruleset>
</patterndb>
</PASTE>

[2]  prior to adding the program, pdbtool just responded with
"example.xml validates", now I get something a bit more verbose.

[nick at localhost ~]$ pdbtool test --validate example.xml
example.xml validates
Testing message program='example' message='Accepted password for
sampleuser from 10.50.0.247 port 42156 ssh2'
 Wrong match name='.classifier.rule_id', value='', expected='182437592347598'
 Wrong match name='SSH.AUTH_METHOD', value='', expected='password'
 Wrong match name='SSH_USERNAME', value='', expected='sampleuser'
 Wrong match name='SSH_CLIENT_ADDRESS', value='', expected='10.50.0.247'
 Wrong match name='SSH_PORT_NUMBER', value='', expected='42156'
[nick at localhost ~]$

[3] I agree that the pattern is wrong, the output above shows that but
the example I'm following if from the balabit documentation, is there
a better reference I should be following?

Following your hint, I've tried changing "<pattern>Accepted
@QSTRING:SSH.AUTH_METHOD: @ for" with "<pattern>Accepted
@ESTRING:SSH.AUTH_METHOD: @for", according to [link*] the "space"
between the final : (colon) and @ (at) should act as a stop char, but
clearly not. Am I trying to run before walking?

[4] I assume you mean "pdbtool patternize -f testfile.log" ? I'm not
sure how that helps...

[nick at localhost ~]$ pdbtool patternize -f testfile.log
[Tue Dec 17 11:57:22 2013] Searching clusters; input lines='4'
[Tue Dec 17 11:57:23 2013] Finding frequent words; phase='caching'
[Tue Dec 17 11:57:23 2013] Finding frequent words; phase='searching'
<patterndb version='3' pub_date='2013-12-17'>
  <ruleset name='patternize' id='8a7a3c95-af22-894b-942b-d4517c389175'>
    <rules>
      <rule id='ed85a798-8440-4044-97e2-ba23753188e5' class='system'
provider='patternize'>
        <!-- support: 2 -->
        <patterns>
          <pattern>password for user from 10.51.0.27 port 4256 ssh2</pattern>
        </patterns>
        <examples>
            <example>
                <test_message program='patternize'>password for user
from 10.51.0.27 port 4256 ssh2</test_message>
            </example>
        </examples>
      </rule>
      <rule id='44834044-fda5-2040-ae58-048bbc039d3d' class='system'
provider='patternize'>
        <!-- support: 2 -->
        <patterns>
          <pattern>password for sampleuser from 10.50.0.247 port 42156
ssh2</pattern>
        </patterns>
        <examples>
            <example>
                <test_message program='patternize'>password for
sampleuser from 10.50.0.247 port 42156 ssh2</test_message>
            </example>
        </examples>
      </rule>
    </rules>
  </ruleset>
</patterndb>
[nick at localhost ~]$

I would have expected pdbtool to create three variables for
'sampleuser' , the IP addresses and port numbers as they are the
things which change on each line of the file. I tried updating
testlogfile to have four unique entries to give it a bit more chance
to spot the changes but still no luck.

Thanks in Advance,
Nick



[link*]http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-3.3-guides/en/syslog-ng-ose-v3.3-guide-admin-en/html/patterndb-using-parsers.html

On 17 December 2013 11:31, Fabien Wernli <wernli at in2p3.fr> wrote:
> Hi,
>
> Your pattern is wrong, and doesn't match your example.
> First of all, add a "program" attribute to your test_message.
> Second, use 'pdbtool' to test your pdb.
> Third, correct your pattern :)
> Fourth, use 'pdbtool' to parse your logfile for you.
>
> Hint: look up the documnentation for QSTRING pattern, and also look at
> @ESTRING
>
> cheers
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>



-- 
Shameless plug for google Juice: http://www.linickx.com