Hi, I’m trying to get a valid json formatted log file. I’ve tried several options but none of them seem to work or parse out the data correctly. Here’s what the syslog-ng config looks like. @version:3.14 @include "scl.conf" # syslog-ng configuration file. # # See syslog-ng(8) and syslog-ng.conf(5) for more information. # # Note: it also sources additional configuration files (*.conf) # located in /etc/syslog-ng/conf.d/ ###################################### #Sources ###################################### source s_netskope { tcp(ip(0.0.0.0) port(51410)); }; ###################################### #Destinations ###################################### destination d_netskope { file("/data/log/syslog/netskope/$HOST/$YEAR-$MONTH-$DAY-netskope.log" create_dirs(yes)); }; ###################################### # Filters ###################################### ###################################### # Log ###################################### log { source( s_netskope); destination(d_netskope); }; options { flush_lines (0); time_reopen (10); log_fifo_size (1000); chain_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); owner("user01"); group("user01"); dir-owner("user01"); dir-group("user01"); dir-perm(0755); perm(0755); }; Any help is greatly appreciated. Thank you, Julio Garcia Pro, Information Security Engineer CoreLogic Direct (949) 214-1284 Mobile (714) 474-5254 julgarcia@corelogic.com<mailto:julgarcia@corelogic.com> corelogic.com<http://www.corelogic.com/> | Blog<http://www.corelogic.com/blog/default.aspx> LinkedIn<http://www.linkedin.com/company/corelogic> | Twitter<http://twitter.com/corelogicinc> | Facebook<http://www.facebook.com/CoreLogic> | Google+<https://plus.google.com/114618839782139347829> Our Vision: Deliver unique property-level insights that power the global real estate economy ****************************************************************************************** This message may contain confidential or proprietary information intended only for the use of the addressee(s) named above or may contain information that is legally privileged. If you are not the intended addressee, or the person responsible for delivering it to the intended addressee, you are hereby notified that reading, disseminating, distributing or copying this message is strictly prohibited. If you have received this message by mistake, please immediately notify us by replying to the message and delete the original message and any copies immediately thereafter. Thank you. ****************************************************************************************** CLLD
Hi, this config file just opens a tcp listener on port 51400 and then writes anything that receives from there into a logfile. I don't really understand what you would like to achieve? On Thu, Dec 20, 2018 at 3:43 PM Garcia, Julio (InfoSec) < julgarcia@corelogic.com> wrote:
Hi, I’m trying to get a valid json formatted log file. I’ve tried several options but none of them seem to work or parse out the data correctly.
Here’s what the syslog-ng config looks like.
@version:3.14
@include "scl.conf"
# syslog-ng configuration file.
#
# See syslog-ng(8) and syslog-ng.conf(5) for more information.
#
# Note: it also sources additional configuration files (*.conf)
# located in /etc/syslog-ng/conf.d/
######################################
#Sources
######################################
source s_netskope {
tcp(ip(0.0.0.0) port(51410));
};
######################################
#Destinations
######################################
destination d_netskope { file( "/data/log/syslog/netskope/$HOST/$YEAR-$MONTH-$DAY-netskope.log" create_dirs(yes)); };
######################################
# Filters
######################################
######################################
# Log
######################################
log { source( s_netskope); destination(d_netskope); };
options {
flush_lines (0);
time_reopen (10);
log_fifo_size (1000);
chain_hostnames (off);
use_dns (no);
use_fqdn (no);
create_dirs (no);
keep_hostname (yes);
owner("user01");
group("user01");
dir-owner("user01");
dir-group("user01");
dir-perm(0755);
perm(0755);
};
Any help is greatly appreciated.
Thank you,
*Julio Garcia* Pro, Information Security Engineer *CoreLogic*
*Direct* (949) 214-1284 *Mobile* (714) 474-5254 *julgarcia@corelogic.com <julgarcia@corelogic.com> *
corelogic.com <http://www.corelogic.com/> | Blog <http://www.corelogic.com/blog/default.aspx> LinkedIn <http://www.linkedin.com/company/corelogic> | Twitter <http://twitter.com/corelogicinc> | Facebook <http://www.facebook.com/CoreLogic> | Google+ <https://plus.google.com/114618839782139347829>
*Our Vision: *Deliver unique property-level insights that power the global real estate economy
******************************************************************************************
This message may contain confidential or proprietary information intended only for the use of the addressee(s) named above or may contain information that is legally privileged. If you are not the intended addressee, or the person responsible for delivering it to the intended addressee, you are hereby notified that reading, disseminating, distributing or copying this message is strictly prohibited. If you have received this message by mistake, please immediately notify us by replying to the message and delete the original message and any copies immediately thereafter.
Thank you. ******************************************************************************************
CLLD
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Hi Julio, If you want to format your logs in JSON, you need to configure the destination accordingly. For instance: destination d_netskope { file( "/data/log/syslog/netskope/$HOST/$YEAR-$MONTH-$DAY-netskope.log" template("$(format-json -s nv-pairs)\n") ); };
I'm now getting the logs in json format but it's all under MESSAGE. Outgoing message; message='{"SOURCE":"s_netskope","PROGRAM":"{\"count\"","MESSAGE":"1, \"supporting_data\": {\"data_values\": [\"Logged out due to inactivity\"], \"data_type\": \"reason\"}, \"organization_unit\": \"XXXXX.XXX/XXXXX/Domain Users/Enterprise Users\", \"severity_level\": 2, \"category\": null, \"timestamp\": 1545331339, \"_insertion_epoch_timestamp\": 1545331340, \"ccl\": \"unknown\", \"user\": \"XXXXX@XXXX.com\", \"audit_log_event\": \"Logout Successful\", \"ur_normalized\": \"XXXXXXX@corelogic.com\", \"_id\": \"c57a37f48db4fa7ca3bef23c\", \"type\": \"admin_audit_logs\", \"appcategory\": null}","LEGACY_MSGHDR":"{\"count\": ","HOST_FROM":"X.X.X.X","HOST":"X.X.X.X"}\x0a' [cid:image001.png@01D49857.F13E0FA0] Thank you, Julio Garcia Pro, Information Security Engineer CoreLogic Direct (949) 214-1284 Mobile (714) 474-5254 julgarcia@corelogic.com corelogic.com <http://www.corelogic.com/> | Blog <http://www.corelogic.com/blog/default.aspx> LinkedIn <http://www.linkedin.com/company/corelogic> | Twitter <http://twitter.com/corelogicinc> | Facebook <http://www.facebook.com/CoreLogic> | Google+ <https://plus.google.com/114618839782139347829> Our Vision: Deliver unique property-level insights that power the global real estate economy On 12/20/18, 6:56 AM, "syslog-ng on behalf of Fabien Wernli" <syslog-ng-bounces@lists.balabit.hu on behalf of wernli@in2p3.fr> wrote: [External Content] This message is from an external source. Please exercise caution when opening attachments or links. Hi Julio, If you want to format your logs in JSON, you need to configure the destination accordingly. For instance: destination d_netskope { file( "/data/log/syslog/netskope/$HOST/$YEAR-$MONTH-$DAY-netskope.log" template("$(format-json -s nv-pairs)\n") ); }; ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq ****************************************************************************************** This message may contain confidential or proprietary information intended only for the use of the addressee(s) named above or may contain information that is legally privileged. If you are not the intended addressee, or the person responsible for delivering it to the intended addressee, you are hereby notified that reading, disseminating, distributing or copying this message is strictly prohibited. If you have received this message by mistake, please immediately notify us by replying to the message and delete the original message and any copies immediately thereafter. Thank you. ****************************************************************************************** CLLD
Hi Julio, On Thu, Dec 20, 2018 at 07:34:18PM +0000, Garcia, Julio (InfoSec) wrote:
I'm now getting the logs in json format but it's all under MESSAGE.
At this point we're gonna need some more information, please: 1. tell us what's sending the data? 2. in what format? 3. send us your complete config file 4. most importantly, explain what exactly you're trying to achieve
participants (3)
-
Fabien Wernli
-
Garcia, Julio (InfoSec)
-
Scheidler, Balázs