On Mon, 2010-10-25 at 19:31 -0400, Worsham, Michael wrote:
We are trying to streamline the syslog-ng v3.x data from the RHEL server-clients to the RHEL DCS/Syslog server while also providing another destination to a local Tripwire Log Center server.
I previously had the version 3.0.8 options (as seen below), which recorded the data files correctly on the DCS server with the correct hostname attribute:
options {
chain_hostnames(no);
time_reopen(10);
time_reap(360);
log_fifo_size(1000);
group(adm);
perm(0644);
dir_perm(0755);
use_dns(yes);
use_fqdn(yes);
create_dirs(yes);
keep_hostname(yes);
log_msg_size(32768);
stats_freq(0);
flush_lines(0);
};
Destination (directory and log file structure):
destination d_general {
file ("/var/log/syslog/general/$HOST/$FULLHOST-$MONTH.$DAY.$YEAR.log"
template("$DATE $HOST <$FACILITY.$PRIORITY> $MSGHDR $MSG\n")
template_escape(no)
);
};
Example: /var/log/syslog/general/CH33Test-WebCO/CH33Test-WebCO-10.25.2010.log
However, we found today that Tripwire Log Center (which we are now required to use for log aggregation/reporting needs) doesn’t have the ability to do hostname translation (i.e. regex the sender’s hostname seen in the datastream and give it an IP address so that it can be tracked). So this means the data file that is being sent from the RHEL clients will need to keep the sender’s IP address in the actual data stream. So if I do keep_hostname(no) and still enable the facility.priority template on the DCS server, it will show the IP address rather than the hostname of the reporting client:
Oct 25 18:51:12 10.153.13.70 <syslog.err> syslog-ng[2820]: EOF occurred while idle; fd='9'
Oct 25 18:51:12 10.153.13.70 <syslog.notice> syslog-ng[2820]: Syslog connection broken; fd='9', server='AF_INET(10.153.29.235:514)', time_reopen='10'
Oct 25 18:51:22 10.153.13.70 <syslog.notice> syslog-ng[2820]: Syslog connection established; fd='9', server='AF_INET(10.153.29.235:514)', local='AF_INET(0.0.0.0:0)'
However, this now causes a problem on the Syslog/DCS server with the local log file storage requirements:
Old Version: /var/log/syslog/general/CH33Test-WebCO/CH33Test-WebCO-10.25.2010.log New Version: /var/log/syslog/general/10.153.13.70/10.153.13.70-10.25.2010.log
NOTE: We don’t have a local DNS that all of the servers can be queried against and that putting in nearly 100+ sender hosts in the /etc/hosts really isn’t a viable option.
How do I satisfy each requirement:
1) Allow the incoming data stream to be saved in a format based on the incoming sender’s hostname for the directory structure (ex: "/var/log/syslog/general/$HOST/$FULLHOST-$MONTH.$DAY.$YEAR.log”)?
2) Allow the IP address of the sender to be seen in the actual data file once saved to the DCS server (ex: “Oct 25 18:51:12 10.153.13.70 <syslog.err>…”)?
3) Is there an option to have the incoming sender’s data saved with both the hostname and IP address in the same header format (ex: “Oct 25 18:51:12 CH33Test-WebCO 10.153.13.70 <syslog.err>…”)?
I've read the complete thread and to say the least I'm confused. Do I understand it right that: 1) you have RHEL clients sending messages to a syslog-ng server 2) you have a syslog-ng server (on DCS) accepting these log messages. you create a file for each client, using the hostname of the client. how does the tripwire agent come into the picture? it reads the files stored on the DCS server? Because if that's the case you can always use $HOST for the filename template and $SOURCEIP as the content template destination d_general { file ("/var/log/syslog/general/$HOST/$FULLHOST-$MONTH.$DAY.$YEAR.log" template("$DATE $SOURCEIP <$FACILITY.$PRIORITY> $MSGHDR $MSG \n") template_escape(no) ); }; This way the content has an IP whereas the filename has the hostname. Is this what you want? Or, since you are already using a non-standard file format (because of the facility/priority values) you can add both, as long as tripwire is able to process it. template("$DATE $SOURCEIP $HOST <$FACILITY.$PRIORITY> $MSGHDR $MSG\n") -- Bazsi