[syslog-ng] Need to parse named query log entries and get just the domain portion

[Robert Webb]

So I have a situation where I have a syslog feed coming in that is sending named (BIND9) query logs. On the DNS servers themselves, we are doing sink hole via a local zone file and sending the dns response back as 127.0.0.1.

Since there is no way for BIND9 to give me logging of the DNS query response back to the user, I need to be able to take a query log entry, parse it to get just the domain portion, and then match the domain to a list. I am trying to do all this inside syslog-ng.

I have been given some guidance to use a pattern database. I worked with this and am successfully matching my incoming query log data. Using the pdbtool, I get a full match and a list of the values I expect. But where I run into an issue after searching through all the documentation is, where can these values be used? I tried building a template against a destination, and none of the values populate. So I am stuck at this point.

I have also been told that I may have to do a custom python parser instead.

Any help or guidance is appreciated.

Here is my rule:

      <rule provider="balabit" id="b0ca071d-309e-454a-8e31-721b01ef55ee" class="system">
        <patterns>
          <pattern>queries: info: client @IPvANY:named.client.ip@#@NUMBER:named.client.port:@ @ESTRING:named.domain.name::@ query: @HOSTNAME:named.host.name:@ @STRING:@ @STRING:named.record.type:@ @ESTRING:named.record.flags: @(@IPvANY:named.server.ip@)</pattern>
        </patterns>
        <examples>
          <example>
            <test_message program="named">client 213.46.246.51#1890: query: tumtali.fegade.com IN A - (103.5.198.211)</test_message>
            <test_values>
              <test_value name="named.client.ip">213.46.246.51</test_value>
              <test_value name="named.client.port">1890</test_value>
              <test_value name="named.domain.name">tumtali.fegade.com</test_value>
              <test_value name="named.record.type">A</test_value>
              <test_value name="named.record.flags">-</test_value>
              <test_value name="named.server.ip">103.5.198.211</test_value>
            </test_values>
          </example>
          <example>
            <test_message program="named">client 10.187.195.186#40441: query: banister.paars.tld IN A -EDC (10.226.134.25)</test_message>
            <test_values>
              <test_value name="named.client.ip">10.187.195.186</test_value>
              <test_value name="named.client.port">40441</test_value>
              <test_value name="named.domain.name">banister.paars.tld</test_value>
              <test_value name="named.record.type">A</test_value>
              <test_value name="named.record.flags">-EDC</test_value>
              <test_value name="named.server.ip">10.226.134.25</test_value>
            </test_values>
          </example>
          <example>
            <test_message program="named">queries: client 192.168.102.189#58454: query: maps.googleapis.com IN A + (192.168.102.2)</test_message>
            <test_values>
              <test_value name="named.client.ip">192.168.102.189</test_value>
              <test_value name="named.client.port">58454</test_value>
              <test_value name="named.domain.name">maps.googleapis.com</test_value>
              <test_value name="named.record.type">A</test_value>
              <test_value name="named.record.flags">+</test_value>
              <test_value name="named.server.ip">192.168.102.2</test_value>
            </test_values>
          </example>
        </examples>
        <values>
        </values>
        <tags>
          <tag>named</tag>
          <tag>queries</tag>
        </tags>
      </rule>




Here are test results:

Pattern matching part:
queries: info: client @IP:named.client.ip=137.75.76.138@#@NUMBER:named.client.port=38643@ @ESTRING:named.domain.name=(sr.symcb.com)@ query: @HOSTNAME:named.host.name=sr.symcb.com@ @STRING:None=IN@ @STRING:named.record.type=A@ @ESTRING:named.record.flags=+E@(@IP:named.server.ip=137.75.75.75@)
Matching part:
queries: info: client 137.75.76.138#38643 (sr.symcb.com): query: sr.symcb.com IN A +E (137.75.75.75)
Values:
MESSAGE=queries: info: client 137.75.76.138#38643 (sr.symcb.com): query: sr.symcb.com IN A +E (137.75.75.75)
PROGRAM=named
.classifier.class=system
.classifier.rule_id=b0ca071d-309e-454a-8e31-721b01ef55ee
named.client.ip=137.75.76.138
named.client.port=38643
named.domain.name=(sr.symcb.com)
named.record.type=A
named.server.ip=137.75.75.75
named.host.name=sr.symcb.com
named.record.flags=+E
TAGS=.classifier.system,named,queries
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20171027/fac70de4/attachment.html>