[ANN]: balabit.logstore - Open Source LogStore reader API
--------------------------------------------------------------- PACKAGE : com.balabit/logstore VERSION : 0.0.1 SUMMARY : First alpha release DATE : 2012 April 24 HOMEPAGE : http://algernon.github.com/balabit.logstore --------------------------------------------------------------- DESCRIPTION: The balabit.logstore project is a library written in Clojure, that tries to provide a convenient API to read syslog-ng PE LogStore files. The reason behind the implementation is to have an independent, open source reader for the LogStore file format, so that one is not tied to syslog-ng to read one's logs stored in this format. An open implementation makes it possible to read these logs on systems where syslog-ng is not installed, or where the lgstool program is not available. LogStore itself is a storage format for log messages, supporting encryption, compression, secure timestamps, serialized messages (think name-value pairs as opposed to just a plain, formatted string) and a lot more. This format is being opened, and this library is meant to provide an implenetation for a LogStore reader, that will eventually support reading all kinds of LogStore files. EXAMPLE: (ns example (:require [balabit.logstore :as logstore])) (defn lgstool-cat ([] (lgstool-cat "resources/logstores/short.compressed.store")) ([filename] (logstore/with-file filename (let [indexed-records (zipmap (range (count (logstore/records))) (logstore/records)) chunk-records (keys (filter #(= :chunk (:type (val %))) indexed-records)) print-msgs (fn [index] (dorun (map println (:messages (logstore/nth-record index)))))] (dorun (map print-msgs chunk-records)))))) (lgstool-cat "/path/to/logstore/messages.store") More examples can be found bundled with the sources, and as part of the documentation: http://algernon.github.com/balabit.logstore/#balabit.logstore.examples FEATURES: The library - being in very early alpha stage - can already read some LogStore files, and transparently decompress them, collect meta-data about both the logstore, and the information stored within. It comes with an extensive test suite and a lot of examples. FUTURE DIRECTION: We are working on making the library able to deserialize name-value pairs, to make it support decrypting logstores, and paralell to that, we're working on a Java API, so that it can be used from a more familiar language, and still provide a convenient, easy to use API. In the future, we also plan to publish ready to use JAR files that one can easily use from any Java or Clojure project. DOWNLOADS: The source is available from the git repository at github: git://github.com/algernon/balabit.logstore.git Documentation and more information about the library is available on its homepage at http://algernon.github.com/balabit.logstore/. People interested in the development of the library can also look at the developer docs at: http://algernon.github.com/balabit.logstore/developer-api.html -- |8]
Hi, Is there any roadmap of the Java API? Balazs On 04/30/2012 01:19 PM, Gergely Nagy wrote:
--------------------------------------------------------------- PACKAGE : com.balabit/logstore VERSION : 0.0.1 SUMMARY : First alpha release DATE : 2012 April 24 HOMEPAGE : http://algernon.github.com/balabit.logstore ---------------------------------------------------------------
DESCRIPTION:
The balabit.logstore project is a library written in Clojure, that tries to provide a convenient API to read syslog-ng PE LogStore files.
The reason behind the implementation is to have an independent, open source reader for the LogStore file format, so that one is not tied to syslog-ng to read one's logs stored in this format. An open implementation makes it possible to read these logs on systems where syslog-ng is not installed, or where the lgstool program is not available.
LogStore itself is a storage format for log messages, supporting encryption, compression, secure timestamps, serialized messages (think name-value pairs as opposed to just a plain, formatted string) and a lot more. This format is being opened, and this library is meant to provide an implenetation for a LogStore reader, that will eventually support reading all kinds of LogStore files.
EXAMPLE:
(ns example (:require [balabit.logstore :as logstore]))
(defn lgstool-cat ([] (lgstool-cat "resources/logstores/short.compressed.store")) ([filename] (logstore/with-file filename (let [indexed-records (zipmap (range (count (logstore/records))) (logstore/records)) chunk-records (keys (filter #(= :chunk (:type (val %))) indexed-records)) print-msgs (fn [index] (dorun (map println (:messages (logstore/nth-record index)))))] (dorun (map print-msgs chunk-records))))))
(lgstool-cat "/path/to/logstore/messages.store")
More examples can be found bundled with the sources, and as part of the documentation: http://algernon.github.com/balabit.logstore/#balabit.logstore.examples
FEATURES:
The library - being in very early alpha stage - can already read some LogStore files, and transparently decompress them, collect meta-data about both the logstore, and the information stored within.
It comes with an extensive test suite and a lot of examples.
FUTURE DIRECTION:
We are working on making the library able to deserialize name-value pairs, to make it support decrypting logstores, and paralell to that, we're working on a Java API, so that it can be used from a more familiar language, and still provide a convenient, easy to use API.
In the future, we also plan to publish ready to use JAR files that one can easily use from any Java or Clojure project.
DOWNLOADS:
The source is available from the git repository at github: git://github.com/algernon/balabit.logstore.git
Documentation and more information about the library is available on its homepage at http://algernon.github.com/balabit.logstore/. People interested in the development of the library can also look at the developer docs at: http://algernon.github.com/balabit.logstore/developer-api.html
Vámos Balázs <vamos.balazs@zuriel.hu> writes:
Hi,
Is there any roadmap of the Java API?
I can't say dates, unfortunately, but the current roadmap consists of finishing up the support for serialized logstores, immediately followed by the Java API. Long story short: Soon(tm). It's one of the most important tasks, and I have half-baked code ready, but I want to finish the other thing first, because I'll have to adjust the Java API after that anyway. -- |8]
participants (2)
-
Gergely Nagy
-
Vámos Balázs