[syslog-ng] format-json reverse order?

[Scheidler, Balázs]

Hi,



-- 
Bazsi

On Mon, Nov 2, 2015 at 6:39 AM, Gergely Nagy <algernon at madhouse-project.org>
wrote:

> On Sat, Oct 31, 2015 at 1:28 PM, Scheidler, Balázs
> <balazs.scheidler at balabit.com> wrote:
> > Hi,
> >
> > I've encountered a case where format-json orders keys not alphabetically,
> > but rather in the other direction. Can you remember any reason for that?
>
> It's for the flat format => structured format conversion. Consider you
> have keys like a.b.c, a.b.d, a.b.e, a.c.a, a.c.f. With reverse
> sorting, you get a.c.f first, and generate an f key, then a, and
> collect that into c. Then you get the b stuff, and then wrap them all
> in a. It may be possible to do it the other way around, generating a
> first, and extending it, but this order is more straightforward in my
> opinion.
>

It seems to have worked by flipping the order, but maybe I wasn't testing
it enough. I don't understand your example, though, "f", then "a", then
"c", the original example was a.c.f

But since there's a definite reason for the ordering, I'm dropping the
reordering patch.



>
> I also seem to remember needing this for array support, but I'm not
> exactly sure of that.
>

this makes sense at least as long as the index is alphabetically sortable
(e.g. the same length), with stuff where indexes contain variable length
numbers (10 sorts before 9), it probably wouldn't work.


>
> As for getting rid of sorting: that'd just make things slower in the
> end, unless syslog-ng starts storing its key-value pairs in an already
> structured format. But even then, my gut feeling is that sorting is
> still faster. If the data is unsorted, you'd have to hunt down where
> to insert the new keys, possibly deep into other structures. That
> sounds very inefficient. Also considerably more code


Yup, sorting is an architectural must at least now.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20151103/b8133811/attachment.htm