[ANN]: balabit.logstore 0.1.0 - Open Source LogStore reader API
--------------------------------------------------------------- PACKAGE : com.balabit/logstore VERSION : 0.1.0 SUMMARY : Rewritten from scratch, now includes simple Java API DATE : 2012 December 15 HOMEPAGE : http://algernon.github.com/balabit.logstore --------------------------------------------------------------- DESCRIPTION: The balabit.logstore project is a library written in Clojure, that tries to provide a convenient API to read syslog-ng PE LogStore files. The reason behind the implementation is to have an independent, open source reader for the LogStore file format, so that one is not tied to syslog-ng to read one's logs stored in this format. An open implementation makes it possible to read these logs on systems where syslog-ng is not installed, or where the lgstool program is not available. LogStore itself is a storage format for log messages, supporting encryption, compression, secure timestamps, serialized messages (think name-value pairs as opposed to just a plain, formatted string) and a lot more. This format is being opened, and this library is meant to provide an implenetation for a LogStore reader, that will eventually support reading all kinds of LogStore files. This release is a complete rewrite of an earlier attempt, and this time, it comes with a simple Java API. EXAMPLE: Clojure: (ns example (:require [balabit.logstore.sweet :as logstore])) (prn (logstore/messages (logstore/from-file "/path/to/logstore/messages.store"))) Java: import BalaBit.LogStore; import clojure.lang.LazySeq; import clojure.lang.Keyword; import java.util.Map; public class LGSCat { public static void main(String[] args) { Keyword k = BalaBit.LogStore.keyword("MESSAGE"); Object o = BalaBit.LogStore.fromFile (args[0]); LazySeq s = (LazySeq) BalaBit.LogStore.messages (o); for (Object m : s.toArray()) { Map msg = (Map) m; System.out.println(msg.get(k)); } } } More examples can be found bundled with the sources, and as part of the documentation: http://algernon.github.com/balabit.logstore/#balabit.logstore.cli FEATURES: The library - while still experimental - can read most unencrypted LogStores, created by syslog-ng PE 4.0 or any later version, and the messages stored within can be explored. It comes with a test suite and quite a few examples. FUTURE DIRECTION: The library is very simple right now, the error handling is pretty much non-existent, and it does not perform any kind of validation. The Java API is very thin, and exposes far more Clojure-isms than it ideally should, this will be improved upon in the future. Once things stabilized a bit, we plan to publish ready to use JAR files that can be used from any Java or Clojure project. DOWNLOADS: The source is available from the git repository at github: git://github.com/algernon/balabit.logstore.git Documentation and more information about the library is available on its homepage at http://algernon.github.com/balabit.logstore/. -- |8]
Gergely Nagy <algernon@balabit.hu> writes:
EXAMPLE:
Clojure:
(ns example (:require [balabit.logstore.sweet :as logstore]))
(prn (logstore/messages (logstore/from-file "/path/to/logstore/messages.store")))
Java:
A short followup on this part: as I wrote in the announcement, the Java API is being constantly improved. So much so, that the below example can now be rewritten much simpler:
import BalaBit.LogStore; import clojure.lang.LazySeq; import clojure.lang.Keyword; import java.util.Map;
public class LGSCat { public static void main(String[] args) { Keyword k = BalaBit.LogStore.keyword("MESSAGE"); Object o = BalaBit.LogStore.fromFile (args[0]); LazySeq s = (LazySeq) BalaBit.LogStore.messages (o);
for (Object m : s.toArray()) { Map msg = (Map) m; System.out.println(msg.get(k)); } } }
..becomes: import BalaBit.LogStoreMap; import BalaBit.LogStore; import java.util.Map; public class LGSCat { public static void main (String[] args) { LogStore lgs = new LogStore (args[0]); for (Object m : lgs.messages ()) { LogStoreMap msg = new LogStoreMap (m); System.out.println (msg.get("MESSAGE")); } } } No more clojure.lang.* imports, no more LazySeq, no more keyword hackery, and everything's neatly tucked away under two simple classes. If there are any Java hackers out there, who would be willing to give me suggestions on how to improve the Java API further, please do let me know! (I barely speak Java, so what I find convenient, may very well not be found as such by others.) I'll iron out a few things, and add ability to do verification (as in, throwing proper exceptions when a LogStore file is found to be broken, instead of hoping that something somewhere will break and throw one for us), then release another version, this time with JARs uploaded aswell. -- |8]
participants (1)
-
Gergely Nagy