WebGUI for Syslog-ng storing logs in HDFS
Hi, I'm looking for a WebGUI application for users to search for syslog, which the logs are stored by syslog-ng into HDFS. Can you please share what WebGUI you're using? I realize ElasticSearch with Kibana are the famous solution for this, however, I wish to store my syslog in HDFS. Thank you. Regards, KK
It's not free - but you might look at the Syslog Store Box - https://www.syslog-ng.com/products/log-management-appliance/ They have also done some cool stuff with a native splunk HEC destination in the professional edition of syslog-ng. Might look at using that and splunk for the front-end? Jim On Fri, Apr 5, 2019 at 2:38 AM Lee Keng Ket <kengket@gmail.com> wrote:
Hi,
I'm looking for a WebGUI application for users to search for syslog, which the logs are stored by syslog-ng into HDFS.
Can you please share what WebGUI you're using?
I realize ElasticSearch with Kibana are the famous solution for this, however, I wish to store my syslog in HDFS.
Thank you.
Regards, KK
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Hi, How about open source webGUI? Any recommendations? Thank you. Regards, KK On Fri, 5 Apr 2019, 20:56 Jim Hendrick, <james.r.hendrick@gmail.com> wrote:
It's not free - but you might look at the Syslog Store Box - https://www.syslog-ng.com/products/log-management-appliance/
They have also done some cool stuff with a native splunk HEC destination in the professional edition of syslog-ng. Might look at using that and splunk for the front-end?
Jim
On Fri, Apr 5, 2019 at 2:38 AM Lee Keng Ket <kengket@gmail.com> wrote:
Hi,
I'm looking for a WebGUI application for users to search for syslog, which the logs are stored by syslog-ng into HDFS.
Can you please share what WebGUI you're using?
I realize ElasticSearch with Kibana are the famous solution for this, however, I wish to store my syslog in HDFS.
Thank you.
Regards, KK
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Hi, Many syslog-ng users store logs to Elasticsearch and search/analyze them using Kibana. Personally I never tested, but should take a look at Elasticsearch-Hadoop if it solves your problem: https://www.elastic.co/products/hadoop Bye, Peter Czanik (CzP) <peter.czanik@balabit.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik On Fri, Apr 5, 2019 at 3:50 PM Lee Keng Ket <kengket@gmail.com> wrote:
Hi,
How about open source webGUI? Any recommendations?
Thank you.
Regards, KK
On Fri, 5 Apr 2019, 20:56 Jim Hendrick, <james.r.hendrick@gmail.com> wrote:
It's not free - but you might look at the Syslog Store Box - https://www.syslog-ng.com/products/log-management-appliance/
They have also done some cool stuff with a native splunk HEC destination in the professional edition of syslog-ng. Might look at using that and splunk for the front-end?
Jim
On Fri, Apr 5, 2019 at 2:38 AM Lee Keng Ket <kengket@gmail.com> wrote:
Hi,
I'm looking for a WebGUI application for users to search for syslog, which the logs are stored by syslog-ng into HDFS.
Can you please share what WebGUI you're using?
I realize ElasticSearch with Kibana are the famous solution for this, however, I wish to store my syslog in HDFS.
Thank you.
Regards, KK
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
I was considering HDFS but ultimately chose ES because of simplicity. I too spent a while looking for a pre-packaged suite but most things are commercial. I'd like to check this out sometime, just haven't had the opportunity - https://opendistro.github.io/for-elasticsearch/ On Fri, Apr 5, 2019 at 10:00 AM Czanik, Péter <peter.czanik@balabit.com> wrote:
Hi,
Many syslog-ng users store logs to Elasticsearch and search/analyze them using Kibana. Personally I never tested, but should take a look at Elasticsearch-Hadoop if it solves your problem: https://www.elastic.co/products/hadoop
Bye,
Peter Czanik (CzP) <peter.czanik@balabit.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik
On Fri, Apr 5, 2019 at 3:50 PM Lee Keng Ket <kengket@gmail.com> wrote:
Hi,
How about open source webGUI? Any recommendations?
Thank you.
Regards, KK
On Fri, 5 Apr 2019, 20:56 Jim Hendrick, <james.r.hendrick@gmail.com> wrote:
It's not free - but you might look at the Syslog Store Box - https://www.syslog-ng.com/products/log-management-appliance/
They have also done some cool stuff with a native splunk HEC destination in the professional edition of syslog-ng. Might look at using that and splunk for the front-end?
Jim
On Fri, Apr 5, 2019 at 2:38 AM Lee Keng Ket <kengket@gmail.com> wrote:
Hi,
I'm looking for a WebGUI application for users to search for syslog, which the logs are stored by syslog-ng into HDFS.
Can you please share what WebGUI you're using?
I realize ElasticSearch with Kibana are the famous solution for this, however, I wish to store my syslog in HDFS.
Thank you.
Regards, KK
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Hi, Peter Thanks for your info, will look into the link you share. As HDFS is decided to be the storage of logs, I'm trying not to store another copy in ES. Hi, Nik What's the difference between this open-distro and the ES from elastic.co? Thank you. Regards, KK On Fri, Apr 5, 2019 at 10:21 PM Nik Ambrosch <nik@ambrosch.com> wrote:
I was considering HDFS but ultimately chose ES because of simplicity. I too spent a while looking for a pre-packaged suite but most things are commercial.
I'd like to check this out sometime, just haven't had the opportunity - https://opendistro.github.io/for-elasticsearch/
On Fri, Apr 5, 2019 at 10:00 AM Czanik, Péter <peter.czanik@balabit.com> wrote:
Hi,
Many syslog-ng users store logs to Elasticsearch and search/analyze them using Kibana. Personally I never tested, but should take a look at Elasticsearch-Hadoop if it solves your problem: https://www.elastic.co/products/hadoop
Bye,
Peter Czanik (CzP) <peter.czanik@balabit.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik
On Fri, Apr 5, 2019 at 3:50 PM Lee Keng Ket <kengket@gmail.com> wrote:
Hi,
How about open source webGUI? Any recommendations?
Thank you.
Regards, KK
On Fri, 5 Apr 2019, 20:56 Jim Hendrick, <james.r.hendrick@gmail.com> wrote:
It's not free - but you might look at the Syslog Store Box - https://www.syslog-ng.com/products/log-management-appliance/
They have also done some cool stuff with a native splunk HEC destination in the professional edition of syslog-ng. Might look at using that and splunk for the front-end?
Jim
On Fri, Apr 5, 2019 at 2:38 AM Lee Keng Ket <kengket@gmail.com> wrote:
Hi,
I'm looking for a WebGUI application for users to search for syslog, which the logs are stored by syslog-ng into HDFS.
Can you please share what WebGUI you're using?
I realize ElasticSearch with Kibana are the famous solution for this, however, I wish to store my syslog in HDFS.
Thank you.
Regards, KK
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
participants (4)
-
Czanik, Péter
-
Jim Hendrick
-
Lee Keng Ket
-
Nik Ambrosch