<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">The source in this case is a fava
application logging with log4j2.<br>
They log to a syslog tcp socket on the local host.<br>
<br>
What I have is a java stack trace that looks like.<br>
<br>
2018-03-20T00:05:00 briard daemon.err iiq1r: ERROR api.Aggregator
- Exception during aggregation. Reason:
java.lang.RuntimeException: sailpoint.tools.GeneralException:
Errors returned from IQService. The changeToken refers to a time
before the start of the current change log.<br>
2018-03-20T00:05:00 briard daemon.err java.lang.RuntimeException:
sailpoint.tools.GeneralException: Errors returned from IQService.
The changeToken refers to a time before the start of the current
change log.<br>
2018-03-20T00:05:00 briard daemon.err at
sailpoint.connector.SharePointRWConnector$SharePointIterator.hasNext(SharePointRWConnector.java:700)<br>
2018-03-20T00:05:00 briard daemon.err at
sailpoint.connector.ConnectorProxy$CustomizingIterator.peek(ConnectorProxy.java:829)<br>
2018-03-20T00:05:00 briard daemon.err at
sailpoint.connector.ConnectorProxy$CustomizingIterator.hasNext(ConnectorProxy.java:856)<br>
2018-03-20T00:05:00 briard daemon.err at
sailpoint.api.Aggregator.aggregateAccounts(Aggregator.java:2799)<br>
2018-03-20T00:05:00 briard daemon.err at
sailpoint.api.Aggregator.primaryAccountAggregation(Aggregator.java:2498)<br>
2018-03-20T00:05:00 briard daemon.err at
sailpoint.api.Aggregator.aggregateApplication(Aggregator.java:2348)<br>
2018-03-20T00:05:00 briard daemon.err at
sailpoint.api.Aggregator.phaseAggregate(Aggregator.java:2250)<br>
2018-03-20T00:05:00 briard daemon.err at
sailpoint.api.Aggregator.execute(Aggregator.java:1868)<br>
2018-03-20T00:05:00 briard daemon.err at
sailpoint.task.ResourceIdentityScan.doUnpartitioned(ResourceIdentityScan.java:219)<br>
2018-03-20T00:05:00 briard daemon.err at
sailpoint.task.ResourceIdentityScan.execute(ResourceIdentityScan.java:199)<br>
2018-03-20T00:05:00 briard daemon.err at
sailpoint.api.TaskManager.runSync(TaskManager.java:796)<br>
2018-03-20T00:05:00 briard daemon.err at
sailpoint.scheduler.JobAdapter.execute(JobAdapter.java:123)<br>
2018-03-20T00:05:00 briard daemon.err at
org.quartz.core.JobRunShell.run(JobRunShell.java:202)<br>
2018-03-20T00:05:00 briard daemon.err at
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)<br>
2018-03-20T00:05:00 briard daemon.err Caused by:
sailpoint.tools.GeneralException: Errors returned from IQService.
The changeToken refers to a time before the start of the current
change log.<br>
2018-03-20T00:05:00 briard daemon.err at
sailpoint.connector.RPCService.checkForErrors(RPCService.java:518)<br>
2018-03-20T00:05:00 briard daemon.err at
sailpoint.connector.RPCService.parseResponse(RPCService.java:445)<br>
2018-03-20T00:05:00 briard daemon.err at
sailpoint.connector.RPCService.execute(RPCService.java:394)<br>
2018-03-20T00:05:00 briard daemon.err at
sailpoint.connector.SharePointRWConnector$SharePointIterator.getNextBlock(SharePointRWConnector.java:608)<br>
2018-03-20T00:05:00 briard daemon.err at
sailpoint.connector.SharePointRWConnector$SharePointIterator.hasNext(SharePointRWConnector.java:663)<br>
2018-03-20T00:05:00 briard daemon.err ... 13 more<br>
<br>
<br>
The first line has the application name, and then all of the
others are really just part of the multi-linem message.
Unfortunately this is arriving on a tcp<br>
socket, which does not support multi-line messages.<br>
<br>
Does log4j2 support syslog protocol?<br>
Does log4j2 support json format?<br>
<br>
That's won't solve my first issue in that the application actually
breaks the messages.<br>
<br>
2018-03-20T00:00:15 briard daemon.debug iiq1r: DEBUG
idam.SyslogStats -
syslogEvents,env=preprod,server=boerboel,className=sailpoint.provisioning.PlanEvaluator,eventLevel=ERROR
count=1 1521529200
syslogEvents,env=preprod,server=boerboel,className=sailpoint.api.Workflower,eventLevel=WARN
count=1 1521529200
syslogEvents,env=preprod,server=boerboel,className=sailpoint.provisioning.PlanEvaluator,eventLevel=ERROR
count=1 1521529200
syslogEvents,env=preprod,server=boerboel,className=sailpoint.connector.LDAPConnector,eventLevel=ERROR
count=1 1521529200
syslogEvents,env=preprod,server=boerboel.comp.uv...<br>
<br>
2018-03-20T00:00:15 briard daemon.debug
...ic.ca,className=sailpoint.scheduler.JobAdapter,eventLevel=ERROR
count=1 1521529200
syslogEvents,env=preprod,server=boerboel,className=sailpoint.provisioning.PlanEvaluator,eventLevel=ERROR
count=1 1521529200
syslogEvents,env=preprod,server=boerboel,className=sailpoint.scheduler.JobAdapter,eventLevel=ERROR
count=1 1521529200
syslogEvents,env=preprod,server=boerboel,className=sailpoint.task.Housekeeper$WorkflowerThread,eventLevel=ERROR
count=1 1521529200
syslogEvents,env=preprod,server=boerboel,className=sailpoint.provisioning.PlanEvaluator,eventLevel=ERR...<br>
<br>
2018-03-20T00:00:15 briard daemon.debug ...OR count=1 1521529200
syslogEvents,env=preprod,server=boerboel,className=sailpoint.connector.LDAPConnector,eventLevel=ERROR
count=1 1521529200
syslogEvents,env=preprod,server=boerboel,className=sailpoint.api.Workflower,eventLevel=ERROR
count=1 1521529200
syslogEvents,env=preprod,server=boerboel,className=sailpoint.request.RequestHandler,eventLevel=WARN
count=1 1521529200
syslogEvents,env=preprod,server=boerboel,className=sailpoint.provisioning.PlanEvaluator,eventLevel=ERROR
count=1 1521529200
syslogEvents,env=preprod,server=boerboel,classN...<br>
<br>
2018-03-20T00:00:15 briard daemon.debug
...ame=sailpoint.provisioning.PlanEvaluator,eventLevel=ERROR
count=1 1521529200
syslogEvents,env=preprod,server=boerboel,className=sailpoint.provisioning.PlanEvaluator,eventLevel=ERROR
count=1 1521529200
syslogEvents,env=preprod,server=boerboel,className=sailpoint.scheduler.JobAdapter,eventLevel=ERROR
count=1 1521529200
syslogEvents,env=preprod,server=boerboel,className=sailpoint.scheduler.JobAdapter,eventLevel=ERROR
count=1 1521529200
syslogEvents,env=preprod,server=boerboel,className=sailpoint.provisioning.PlanEvaluator,eventLevel=ERROR
count=1 1521529200<br>
<br>
<br>
I will follow up with out java group to see what options are
available to us..<br>
<br>
<br>
On 03/20/2018 06:56 AM, Nagy, Gábor wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAETAYnBYTYzMR_SrAqr3mn9dpxeUrNewf4gbGOo970_HGn4VPA@mail.gmail.com">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<div dir="ltr">I see that the complexity of that regex expression
would increase hugely if you want to solve.
<div><br>
</div>
<div>I'm still thinking about other possibilities before
focusing on a patterndb solution.</div>
<div>What kind of source do you use for that application? Where
is it logging to?</div>
<div><br>
</div>
<div>Gabor</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Tue, Mar 20, 2018 at 2:19 PM, Evan
Rempel <span dir="ltr"><<a href="mailto:erempel@uvic.ca"
target="_blank" moz-do-not-send="true">erempel@uvic.ca</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div class="m_5055162390551485917moz-cite-prefix">No
problem about my name. My fast fingers make tonnes of
errors.<br>
<br>
The application does not log into a file, so that isn't
a really good option.<br>
I have the patterndb working for this, however, I came
across another line that is<br>
<br>
... 20 more<br>
<br>
and has a continuation line preceding it that does NOT
end in ... so I have filter<br>
that one out.<br>
<br>
Does anyone handle java stack dumps gracefully :-)<br>
<br>
Evan
<div>
<div class="h5"><br>
<br>
On 03/20/2018 06:07 AM, Nagy, Gábor wrote:<br>
</div>
</div>
</div>
<blockquote type="cite">
<div>
<div class="h5">
<div dir="ltr">Sorry Evan for mistyping your name.
:)</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Tue, Mar 20, 2018 at
2:06 PM, Nagy, Gábor <span dir="ltr"><<a
href="mailto:gabor.nagy@balabit.com"
target="_blank" moz-do-not-send="true">gabor.nagy@balabit.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0
0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div dir="ltr">Hi Elen!<br>
<br>
Does your application log into a file?
Because then you could use multi-line file
source with a well-defined prefix as the "<span
style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">{date}
{host} {program}:</span>".
<div><br>
</div>
<div>Regards,</div>
<div>Gabor</div>
</div>
<div class="m_5055162390551485917HOEnZb">
<div class="m_5055162390551485917h5">
<div class="gmail_extra"><br>
<div class="gmail_quote">On Thu, Mar 15,
2018 at 7:10 AM, Scheidler, Balázs <span
dir="ltr"><<a
href="mailto:balazs.scheidler@balabit.com"
target="_blank"
moz-do-not-send="true">balazs.scheidler@balabit.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div dir="auto">The <span
class="m_5055162390551485917m_-4500865000135016209m_5465107750960201948m_8211379849010350452money">$1</span> is
not set in this case, you can
however use template functions in
the value part. E.g. set line
based on the @PCRE@ matcher and
overwrite its value using an
expression $(substr $line 0 -3)
<div dir="auto"><br>
</div>
<div dir="auto">Would that work
for you?</div>
<div dir="auto"><br>
</div>
</div>
<div
class="m_5055162390551485917m_-4500865000135016209HOEnZb">
<div
class="m_5055162390551485917m_-4500865000135016209h5">
<div class="gmail_extra"><br>
<div class="gmail_quote">On
Mar 15, 2018 02:08, "Evan
Rempel" <<a
href="mailto:erempel@uvic.ca"
target="_blank"
moz-do-not-send="true">erempel@uvic.ca</a>>
wrote:<br type="attribution">
<blockquote
class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">I
have a case where an
application logs something
like<br>
<br>
{date} {host} {program}:
my first line...<br>
...my second line...<br>
...and my third line.<br>
<br>
<br>
I want to make a
correlation and unwrap
these lines into<br>
<br>
{date} {host} {program}:
my first line my second
line and my third line.<br>
<br>
<br>
I started writing the
patterndb to do this, but
matching the ... at the
end<br>
<br>
of the line is difficult,
so I used
@PCRE:line:(.*)\.\.\.$@<br>
<br>
but I then need to only
use the $1 to set a value<br>
<br>
<values><br>
<value
name="mymessage">$1</value><br>
</values><br>
<br>
<br>
Would this be the correct
syntax to do this?<br>
<br>
Is there an easier way
that would perform well?<br>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</blockquote>
</div>
</div>
</blockquote>
<br>
<pre class="moz-signature" cols="500">--
Evan</pre>
</body>
</html>