[syslog-ng] Using macro substring as another macro

[Gergely Nagy]

Jakub Jankowski <shasta at toxcorp.com> writes:

> Friday 08 of April 2011 00:51:23 Balazs Scheidler napisał(a):
>> On Thu, 2011-04-07 at 15:01 +0200, Jakub Jankowski wrote:
>> > I'd like to implement simple hashing in file() destination, mainly to
>> > avoid ext3 32k directories limit, but also for convenience. Is it
>> > possible to do something simliar what this shell snippet does:
>> > $ NAME=value
>> > $ echo ${NAME[*]:0:1}
>> > v
>> > $
>> > but for macros (especially those coming from parsers)?
> [...]
>> hmm.... thinking out loud. there are basically two syntactic options:
>>
>> 1) to use the new template function syntax
>
> I had a look at the code yesterday, using Gergely's pointers from the other 
> mail (thanks for that!), but one thing stopped me from doing some pointer 
> arithmetics there myself - I wasn't sure how it'd work for unicode (i.e. 
> multibyte) strings. Also pity that glib doesn't have a g_substr() :)

I wouldn't worry about unicode for now, we could call it $(cut), as
Bazsi suggested, and document that it workes on bytes treams, and has no
knowledge of unicode. For the vast majority of cases, that's good enough
in my opinion.

I couldn't think of a case where I would want to split unicode strings
in template functions.

(Also, there's g_strstr_len() in glib, which is almost like substr)

>> 2) to continue extending our bash-like expansion syntax.
>>
>> The 1) option would be something like:
>>
>> $(cut -c 1-10 $NAME)
>>
>> or
>>
>> $(substr $NAME 1 10)
>
> As for the syntax - I'd stick with substr <string> <offset> <length>, just 
> like Perl, PHP, or MySQL do.

$(substr $NAME 1 10) is also easier to parse (and syslog-ng does that
for us to begin with) than -c 1-10 (which would require a second parse),
so I'd vote for the latter too.

>> The alternative would be to add really simple functions as an addition
>> to the basic template syntax. We currently have the "default" values
>> like this way:
>>
>> ${NAME:-default}
>>
>> this results in "default" if $NAME is not-defined or empty.
>>
>> The template function stuff is easier to write (as you can write plugins
>> for that, and there's a whole framework to support it). The bash
>> syntax-like stuff is more difficult to write as the template parser is
>> manually coded, and each expansion mode has to be open-coded, although
>> it has better performance (since we can avoid a copy of $NAME into a
>> temporary buffer).
>
> Efficiency is one of the factors I need to consider, so I'd love to see that 
> ${macro:offset:len} working eventually, but for a starter, having it even as 
> a (slower) template function would be neat too. Do you think syslog-ng could 
> have both implemented $someday?

${macro:offset:len} sounds like a good candidate for a HOWTO!

But, I'll knock up $(substr) first (though, due to work commitments,
that'll be done later tonight, later than I've planned).

I'll post the patch in this thread once it's ready. My offer still
stands, nevertheless!

-- 
|8]