I'm relatively new to RFC5424 and was
under the impression that all of the syslog-ng parsed values
could/should be sent using the
structured data mechanism. This should apply to all macros, not
just TAGS.
In general, what I am proposing is that any "work" done by
syslog-ng on one computer, should be able to be maintained as the
event gets passed
to another syslog-ng computer. The work should not be lost just
because the payload has moved from one system to another.
I was hoping to use the syslog-ng pattern database to parse
messages at the source, matching source ip addresses, user names
etc.
and forward that vi RFC5424 to a syslog-ng system that would then
feed elastic search, passing all of the parsed values to elastic
search for indexing.
If that could not be done by RFC5424, I could use a raw JSON
payload, and unwrap it at the receiving end, however, the source
HOST, TAGS, DATE, TIME etc
are all lost in this case as well.
For the json parser, I think it would be good to have some kind of
option for permitting core macros to be replaced/overwritten.
In the case of TAGS, which is a little bit special in the json
object because it is converted to a string, rather than a json
list, it should be appended to.
Evan.
On 09/03/2015 09:42 PM, Scheidler, Balázs wrote:
you are right, it is a huge oversight. can you pls
suggest an on wire format how this should work?
--
Bazsi
This is a huge
oversight. I have "complained" about this before. A JSON
source (json parser) should append all of the tags from the JSON
payload
into the current set of TAGS.
I'm not sure about syslog protocol (new RFC) if the TAGS is
prepended
with the .SDATA if the syslog parser will populate the TAGS. I
would
hope so.
Evan.
On 09/02/2015 12:00 AM, Fabien Wernli wrote:
> Hi Balázs,
>
> On Wed, Sep 02, 2015 at 07:16:32AM +0200, Scheidler,
Balázs wrote:
>> The best solution to send dara over the wire between
two Syslog-ng
>> instances (e.g. the one getting the logs and the other
storing them in
>> elastic) is to use json to encode name-value pairs.
> That's another way, indeed. What these have in common,
though, is that there
> is no way to transmit TAGS from one syslog-ng instance to
another properly
> (then use tags() filters on the remote end)
>
>
______________________________________________________________________________