All, I have Cloudflare Workers sending log messages to my syslog-ng server. There is an intermediary step where nginx receives a POST with a JSON body containing all the messages for a given run (try{}catch(e){} ensures it sends the accumulated messages at the end of each run). Some Lua in the nginx server parses the JSON and separates the array of messages into individual writes to the unix domain socket syslog-ng is listening on. This whole setup is tested, and working great. I even get the stack trace (JS, not a core dump) when I screw up. :-) Now, I'd like to auto-fire an email on certain events. e.g. when .json.level == "CRIT" (script threw an error). The trick is, I'd like to dump *all* the log messages for the matching run (only a single line from a run might match) to the email / process destination. See the commented out alert_parser{}, below. The last message of every run (equivalent to a unique .json.rayid, which I treat as a process id) always starts with 'BAIL'. So I use that as a trigger. I've been using $searchengine[*] and not been able to figure out how to send /all/ of the messages in a group to aggregate(). I even dug into the source a bit and saw that I can reference messages by '@2' for the second message back. But I see no way to a) get the number of messages, b) loop through the messages, or c) reference all of them, e.g. '@*'. Has anyone solved this problem? tia, Jason. ----------->8--------------------------------------------------------- @version: 3.22 # common parser nginx-lua-parser { json-parser (prefix(".json.")); }; #parser alert_parser { # grouping-by( # key("${json.rayid}") # scope("process") # timeout(5) # having("CRIT") # trigger("BAIL") # aggregate( # value("MESSAGE" "\n\n") # inherit-mode("context") # ) # inject-mode("pass-through") # ); #}; # template nginx-lua-template "${.json.timestamp} ${.json.colo} ${.json.script}[${.json.rayid}]: ${.json.level} ${.json.message}\n"; source worker-src { unix-stream("/var/run/nginx-lua/worker.sock", group(nginx) flags(no-parse)); }; # development logs filter worker-dev-filter {match("-dev" value (".json.script"));}; destination worker-dev-dest { file("/var/log/worker/development.log" template(nginx-lua-template)); }; log { source(worker-src); parser(nginx-lua-parser); filter(worker-dev-filter); destination(worker-dev-dest); }; # production logs filter worker-prod-filter {match("-prod" value (".json.script"));}; destination worker-prod-dest { file("/var/log/worker/production.log" template(nginx-lua-template)); }; log { source(worker-src); parser(nginx-lua-parser); filter(worker-prod-filter); destination(worker-prod-dest); };