[syslog-ng] RFC3339 timestamp parsing

[Balazs Scheidler]

On Thu, 2010-03-04 at 11:24 -0800, Ian Wienand wrote:
> Hi,
> 
> I notice that in the syslog-ng from my Debian install (2.0.9)
> log_msg_parse_date has
> 
>    p = strptime(self->date.str, "%Y-%m-%dT%H:%M:%S", &tm);
> 
> and in 3.1beta2 I see
> 
>    p = (guchar *) strptime((gchar *) date, "%Y-%m-%d T%H:%M:%S", &tm);
> 
> Is the extra space intended?
> 
> I'm also wondering about a seeming ambiguity in RFC3339 [1]
> 
> In 5.6 it says
> 
>    NOTE: ISO 8601 defines date and time separated by "T".  Applications
>    using this syntax may choose, for the sake of readability, to
>    specify a full-date and full-time separated by (say) a space
>    character.
> 
> However in Appendix A it says
> 
>    ISO 8601 states that the "T" may be omitted under some
>    circumstances.  This grammar requires the "T" to avoid ambiguity.
> 
> So is the T required or not?  Should syslog-ng be able to parse either
> format?
> 
> -i
> 

That space got added because of portability reasons, AFAIR it was HP-Ux
which didn't like %dT as a format string. In strftime a space is any
number of white space characters, and can be none. And before parsing
the date with strptime() syslog-ng checks certain characters in the
input string to decide which strptime() format is to be used. And there
the 'T' is required.

  if (left >= 19 && src[4] == '-' && src[7] == '-' && src[10] == 'T' && src[13] == ':' && src[16] == ':')
    {
      /* RFC3339 timestamp, expected format: YYYY-MM-DDTHH:MM:SS[.frac]<+/->ZZ:ZZ */
      ...
    }

If you look at the unit tests under tests/unit/test_msgparse.c you can
see which format the code is tested against.

Currently we don't support timestamps without the 'T' letter, but as the
date recognition is based on heuristics that 'T' is quite important
right now. This could be less important with the new RFC5424 protocol
format, where the timestamp is required to be RFC3339.


-- 
Bazsi