Re: [syslog-ng] Directory/file create ownership
Add create_dirs (yes) in the options section. On 27-Feb-2018 2:24 PM, "Machiel Richards" <machiel.richards@gmail.com> wrote: Good day guys I am having an issue that I seem not be able to resolve thus far. I am fairly new to syslog-ng and have setup a new syslog server. We have managed to get most of our servers sending logs to the server without issues, however I am having trouble to get syslog-ng to create the directories and files with the correct ownership. We have a seperate user and group which it needs to create directories with, however it keeps on creating directories and files as root user. below is my current config : # First, set some global options. options { chain_hostnames(off); flush_lines(0); use_dns(yes); use_fqdn(yes); owner("syslog"); group("syslog"); perm(0640); stats_freq(0); bad_hostname("^gconfd$"); log_fifo_size(10000); }; source inputs { system(); internal(); unix-stream("/dev/log"); tcp(max_connections(50)); udp(); }; destination logpile { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$FACILITY.$YEAR$ MONTH$DAY" owner("syslog") group("syslog") perm(0660) create_dirs(yes) dir_perm(0770)); }; log { source(inputs); destination(logpile); }; Can someone please assist me as any help would be appreciated. Regard -- I am a bomb technician...If you see me running, TRY TO KEEP UP! ____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Hi, I have added that and restarted syslog-ng, however still no difference. Still creates the directories and files with root as owner and group. drwxrwx--- 3 root root 4.0K Feb 27 13:07 On Tue, Feb 27, 2018 at 1:05 PM, Dwijadas Dey <dwijad@gmail.com> wrote:
Add create_dirs (yes) in the options section.
On 27-Feb-2018 2:24 PM, "Machiel Richards" <machiel.richards@gmail.com> wrote:
Good day guys
I am having an issue that I seem not be able to resolve thus far.
I am fairly new to syslog-ng and have setup a new syslog server.
We have managed to get most of our servers sending logs to the server without issues, however I am having trouble to get syslog-ng to create the directories and files with the correct ownership.
We have a seperate user and group which it needs to create directories with, however it keeps on creating directories and files as root user.
below is my current config :
# First, set some global options. options { chain_hostnames(off); flush_lines(0); use_dns(yes); use_fqdn(yes); owner("syslog"); group("syslog"); perm(0640); stats_freq(0); bad_hostname("^gconfd$"); log_fifo_size(10000); };
source inputs { system(); internal(); unix-stream("/dev/log"); tcp(max_connections(50)); udp(); };
destination logpile { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$FACILITY.$YEAR$MONT H$DAY" owner("syslog") group("syslog") perm(0660) create_dirs(yes) dir_perm(0770)); };
log { source(inputs); destination(logpile); };
Can someone please assist me as any help would be appreciated.
Regard
-- I am a bomb technician...If you see me running, TRY TO KEEP UP!
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product= syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- I am a bomb technician...If you see me running, TRY TO KEEP UP!
Hello! Just a guess, but have you tried dir_group, etc. options as well? Regards, Gabor On Tue, Feb 27, 2018 at 12:08 PM, Machiel Richards <machiel.richards@gmail.com> wrote:
Hi,
I have added that and restarted syslog-ng, however still no difference.
Still creates the directories and files with root as owner and group.
drwxrwx--- 3 root root 4.0K Feb 27 13:07
On Tue, Feb 27, 2018 at 1:05 PM, Dwijadas Dey <dwijad@gmail.com> wrote:
Add create_dirs (yes) in the options section.
On 27-Feb-2018 2:24 PM, "Machiel Richards" <machiel.richards@gmail.com> wrote:
Good day guys
I am having an issue that I seem not be able to resolve thus far.
I am fairly new to syslog-ng and have setup a new syslog server.
We have managed to get most of our servers sending logs to the server without issues, however I am having trouble to get syslog-ng to create the directories and files with the correct ownership.
We have a seperate user and group which it needs to create directories with, however it keeps on creating directories and files as root user.
below is my current config :
# First, set some global options. options { chain_hostnames(off); flush_lines(0); use_dns(yes); use_fqdn(yes); owner("syslog"); group("syslog"); perm(0640); stats_freq(0); bad_hostname("^gconfd$"); log_fifo_size(10000); };
source inputs { system(); internal(); unix-stream("/dev/log"); tcp(max_connections(50)); udp(); };
destination logpile {
file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$FACILITY.$YEAR$MONTH$DAY" owner("syslog") group("syslog") perm(0660) create_dirs(yes) dir_perm(0770)); };
log { source(inputs); destination(logpile); };
Can someone please assist me as any help would be appreciated.
Regard
-- I am a bomb technician...If you see me running, TRY TO KEEP UP!
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- I am a bomb technician...If you see me running, TRY TO KEEP UP!
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
ok, that did the trick, thank you very much. On Tue, Feb 27, 2018 at 1:12 PM, Nagy, Gábor <gabor.nagy@balabit.com> wrote:
Hello!
Just a guess, but have you tried dir_group, etc. options as well?
Regards, Gabor
On Tue, Feb 27, 2018 at 12:08 PM, Machiel Richards <machiel.richards@gmail.com> wrote:
Hi,
I have added that and restarted syslog-ng, however still no difference.
Still creates the directories and files with root as owner and group.
drwxrwx--- 3 root root 4.0K Feb 27 13:07
On Tue, Feb 27, 2018 at 1:05 PM, Dwijadas Dey <dwijad@gmail.com> wrote:
Add create_dirs (yes) in the options section.
On 27-Feb-2018 2:24 PM, "Machiel Richards" <machiel.richards@gmail.com> wrote:
Good day guys
I am having an issue that I seem not be able to resolve thus far.
I am fairly new to syslog-ng and have setup a new syslog server.
We have managed to get most of our servers sending logs to the server without issues, however I am having trouble to get syslog-ng to create the directories and files with the correct ownership.
We have a seperate user and group which it needs to create directories with, however it keeps on creating directories and files as
root
user.
below is my current config :
# First, set some global options. options { chain_hostnames(off); flush_lines(0); use_dns(yes); use_fqdn(yes); owner("syslog"); group("syslog"); perm(0640); stats_freq(0); bad_hostname("^gconfd$"); log_fifo_size(10000); };
source inputs { system(); internal(); unix-stream("/dev/log"); tcp(max_connections(50)); udp(); };
destination logpile {
file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$FACILITY.$YEAR$MONTH$DAY" owner("syslog") group("syslog") perm(0660) create_dirs(yes) dir_perm(0770)); };
log { source(inputs); destination(logpile); };
Can someone please assist me as any help would be appreciated.
Regard
-- I am a bomb technician...If you see me running, TRY TO KEEP UP!
____________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
____________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- I am a bomb technician...If you see me running, TRY TO KEEP UP!
____________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- I am a bomb technician...If you see me running, TRY TO KEEP UP!
Hi all, Just to add on this topic, I noticed that when using the docker container all the files and folders gets a permission of root:root despite of what's in the global config options (I guess because the container by default starts with the root user). Did you have any experience with this and do you know how to allow syslog-ng to set the users/groups as per the config options specified? Thanks and sorry if this is slightly off topic... but not that much in effect. Thanks, Marco
Hello Marco, I tested the settings with docker, and the following configuration worked for me. Created a test user/group on my *host* machine: $useradd -M -u 1500 marco Started a docker container: $docker run -it --network=host -v /tmp/marco:/tmp/marco balabit/syslog-ng-xenial bash Input: $cat marco_in.txt Hello World! Configuration: $cat marco.conf @version: 3.14 @include "scl.conf" source s_local { file( "/tmp/marco/marco_in.txt" flags(no-parse) ); }; destination d_logs { file( "/tmp/marco/marco_out.txt" owner(1500) group(1500) perm(0200) ); }; log { source(s_local); destination(d_logs); }; Running syslog-ng: root@tp:/tmp/marco# syslog-ng -Fdev -f marco.conf ls -hal on the host machine: --w------- 1 marco marco 32 márc 2 13:15 marco_out.txt Output on the *host*: $cat marco_out.txt cat: marco_out.txt: Permission denied Output on the *host*: $ sudo cat marco_out.txt Mar 2 12:15:01 tp Hello World! Note: I used the UID and the GID in the syslog-ng configuration file since there is no such user/group inside my docker container. I also tested a scenario where I use the username/groupname of the external user, but of course in this case I have to create them with the same ID inside my container. (If I remember correctly docker provides some basic mapping between external/internal users, but I am not sure about it.) Br, Laci On Thu, Mar 1, 2018 at 7:16 PM, Marco Mignone <info@marcomignone.com> wrote:
Hi all, Just to add on this topic, I noticed that when using the docker container all the files and folders gets a permission of root:root despite of what's in the global config options (I guess because the container by default starts with the root user).
Did you have any experience with this and do you know how to allow syslog-ng to set the users/groups as per the config options specified?
Thanks and sorry if this is slightly off topic... but not that much in effect.
Thanks, Marco
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Hi Laci, Thanks for this. I will have a play at this and I probably need to study a bit more of Docker as my confusion probably derives from the fact of using docker-compose to start all services instead of 'docker run' when one can specify also the user you want the container to start with. The one thing I don't understand is why you can't access the file on the host machine (unless of using sudo) if the user on the host and inside the container are the same? That is basically what I am trying to achieve, the output folder and files to have the ownership of an existing user / group on the host machine so that they are accessible by that user without having to sudo. I wonder if that is what you meant at the end talking about the external user in the 'note:' section of your reply? Thanks for your help and reply, that's already a good starting point for me to try again. Regards, Marco
Hello Marco, please find my answers inline Br, Laci On Sun, Mar 4, 2018 at 4:56 PM, Marco Mignone <info@marcomignone.com> wrote:
Hi Laci, Thanks for this. I will have a play at this and I probably need to study a bit more of Docker as my confusion probably derives from the fact of using docker-compose to start all services instead of 'docker run' when one can specify also the user you want the container to start with.
From what I have found, while the command line interface do not support the *user* parameter, the compose files do. Not so flexible, but fair enough.
The one thing I don't understand is why you can't access the file on the host machine (unless of using sudo) if the user on the host and inside the container are the same?
That was just a small trick to demonstrate that access rights are in place. I forgot to copy the whole command prompt, but on my personal computer I am using the username *szemere*. So with the permission *0200* (seen by *ls -hal*) even I was unable to access the files belonging to the user *marco* (id: *1500*).
That is basically what I am trying to achieve, the output folder and files to have the ownership of an existing user / group on the host machine so that they are accessible by that user without having to sudo. I wonder if that is what you meant at the end talking about the external user in the 'note:' section of your reply?
You are right. By external user I meant the user on the host machine. However my note was about how to address them. The problem: Since your "external" users do not exists (by default) inside the container, you can not use their name to "address" them. (You most probably got a "no such user" error.) The most common solutions to this problem are: A) Select users/groups by their ID. (See in the syslog-ng's configuration in the example.) B) First create the users/groups inside the container with a matching ID. After that you can use the "names" in the syslog-ng's configuration. C) Blind mount the /etc/passwd file. (Has some other implications, read carefully, test with virtual machines before using it.)
Thanks for your help and reply, that's already a good starting point for me to try again.
Regards, Marco ____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Hey Laci, That's brilliant and amazing response! Thanks for all the insight, it will save me from a lot of work and confusion I was having with some docker concepts. Thanks a lot again! Marco
On 5 Mar 2018, at 19:11, Szemere, László <laszlo.szemere@balabit.com> wrote:
Hello Marco, please find my answers inline
Br, Laci
On Sun, Mar 4, 2018 at 4:56 PM, Marco Mignone <info@marcomignone.com <mailto:info@marcomignone.com>> wrote: Hi Laci, Thanks for this. I will have a play at this and I probably need to study a bit more of Docker as my confusion probably derives from the fact of using docker-compose to start all services instead of 'docker run' when one can specify also the user you want the container to start with. From what I have found, while the command line interface do not support the user parameter, the compose files do. Not so flexible, but fair enough.
The one thing I don't understand is why you can't access the file on the host machine (unless of using sudo) if the user on the host and inside the container are the same? That was just a small trick to demonstrate that access rights are in place. I forgot to copy the whole command prompt, but on my personal computer I am using the username szemere. So with the permission 0200 (seen by ls -hal) even I was unable to access the files belonging to the user marco (id: 1500).
That is basically what I am trying to achieve, the output folder and files to have the ownership of an existing user / group on the host machine so that they are accessible by that user without having to sudo. I wonder if that is what you meant at the end talking about the external user in the 'note:' section of your reply? You are right. By external user I meant the user on the host machine. However my note was about how to address them. The problem: Since your "external" users do not exists (by default) inside the container, you can not use their name to "address" them. (You most probably got a "no such user" error.)
The most common solutions to this problem are: A) Select users/groups by their ID. (See in the syslog-ng's configuration in the example.) B) First create the users/groups inside the container with a matching ID. After that you can use the "names" in the syslog-ng's configuration. C) Blind mount the /etc/passwd file. (Has some other implications, read carefully, test with virtual machines before using it.)
Thanks for your help and reply, that's already a good starting point for me to try again.
Regards, Marco ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng <https://lists.balabit.hu/mailman/listinfo/syslog-ng> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng <http://www.balabit.com/support/documentation/?product=syslog-ng> FAQ: http://www.balabit.com/wiki/syslog-ng-faq <http://www.balabit.com/wiki/syslog-ng-faq>
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
participants (5)
-
Dwijadas Dey
-
Machiel Richards
-
Marco Mignone
-
Nagy, Gábor
-
Szemere, László