Hi,

Docker info output

root@hdata3:/home/smb/code/conf# docker info

Containers: 0

 Running: 0

 Paused: 0

 Stopped: 0

Images: 0

Server Version: 18.05.0-ce

Storage Driver: devicemapper

 Pool Name: docker-8:1-100670142-pool

 Pool Blocksize: 65.54kB

 Base Device Size: 10.74GB

 Backing Filesystem: xfs

 Udev Sync Supported: true

 Data file: /dev/loop0

 Metadata file: /dev/loop1

 Data loop file: /var/lib/docker/devicemapper/devicemapper/data

 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata

 Data Space Used: 11.73MB

 Data Space Total: 107.4GB

 Data Space Available: 51.36GB

 Metadata Space Used: 581.6kB

 Metadata Space Total: 2.147GB

 Metadata Space Available: 2.147GB

 Thin Pool Minimum Free Space: 10.74GB

 Deferred Removal Enabled: true

 Deferred Deletion Enabled: true

 Deferred Deleted Device Count: 0

 Library Version: 1.02.146-RHEL7 (2018-01-22)

Logging Driver: json-file

Cgroup Driver: cgroupfs

Plugins:

 Volume: local

 Network: bridge host macvlan null overlay

 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog

Swarm: inactive

Runtimes: runc

Default Runtime: runc

Init Binary: docker-init

containerd version: 773c489c9c1b21a6d78b5c538cd395416ec50f88

runc version: 4fc53a81fb7c994640722ac585fa9ca548971871

init version: 949e6fa

Security Options:

 seccomp

  Profile: default

Kernel Version: 3.10.0-693.21.1.el7.x86_64

Operating System: CentOS Linux 7 (Core)

OSType: linux

Architecture: x86_64

CPUs: 2

Total Memory: 7.147GiB

Name: api-es-syslog-ng-2

ID: HOBZ:AVPN:MRVO:JAMW:YZFU:SF5M:LTFF:7H7Y:BKBA:MNV4:OMY3:4IJM

Docker Root Dir: /var/lib/docker

Debug Mode (client): false

Debug Mode (server): false

Registry: https://index.docker.io/v1/

Labels:

 provider=google

Experimental: false

Insecure Registries:

 127.0.0.0/8

Live Restore Enabled: false

WARNING: devicemapper: usage of loopback devices is strongly discouraged for production use.

         Use `--storage-opt dm.thinpooldev` to specify a custom block storage device.

On 25 May 2018 at 18:21, László Várady <laszlo.varady@balabit.com> wrote:

Hi,

Thanks.

Could you share what version of Docker you are using?

The output of "docker version" and "docker info" might be interesting.

--

László Várady

On Fri, May 25, 2018 at 6:41 PM Mujeeb Baig <baig.mujeeb@gmail.com> wrote:

Hi Laci

Here is the output, this time I created another folder /demo/syslog-ng and copied the syslog-ng.conf file there, but same issue. I had also attached the output of
docker volume inspect syslog-ng  (see attcahment)

By the way thanks for your help so far.

root@hdata3:/home/smb/code/conf# docker rmi balabit/syslog-ng:latest

Untagged: balabit/syslog-ng:latest

Untagged: balabit/syslog-ng@sha256:55ed29d3f68e6ce5c0494580af0b49bdfbbd83232902d9db0596486dbc2f0d3f

Deleted: sha256:aea40146388e85ca11d2b8b4ee7877d2939a71bed41b727431d48b21084967e1

Deleted: sha256:b97aefa7a738770d348b4c7b5ba8d6cfcaada9453e41603db7b57b01a6bf6773

Deleted: sha256:11aa7498ed3edbf74eea2ee5de418fb8789055e278e750d1db5e4fe9057cada4

Deleted: sha256:a15ea7dac472320a77dfffe1066680e7dfd46eea735376a5b8a2040f4d0ca57e

Deleted: sha256:f2f7b467bea911acac35d8a24c3b14cf163bb00370209f5214e5950813d45686

Deleted: sha256:42a1952fea491565f3b8fb19d5869576c8d12a9192f867c4b72234349fa9059a

Deleted: sha256:e30891f348eea2928d4027527455904b5933426cbbef4be4b9a908c97a09ba68

Deleted: sha256:ffc4c11463ee21b7532b63abd6079393c619a5d0f4b00397a4b9d1cf9efc4d9b

root@hdata3:/home/smb/code/conf#

root@hdata3:/home/smb/code/conf# docker pull balabit/syslog-ng

Using default tag: latest

latest: Pulling from balabit/syslog-ng

4176fe04cefe: Pull complete

af68cba418f8: Pull complete

9b2b4fba2935: Pull complete

6297af76ec77: Pull complete

7bf2bd8c0da2: Pull complete

85f1f826e469: Pull complete

24a506d1872d: Pull complete

Digest: sha256:55ed29d3f68e6ce5c0494580af0b49bdfbbd83232902d9db0596486dbc2f0d3f

Status: Downloaded newer image for balabit/syslog-ng:latest

root@hdata3:/home/smb/code/conf#

root@hdata3:/home/smb/code/conf#

root@hdata3:/home/smb/code/conf# docker run -it -v /demo/syslog-ng/syslog-ng.conf:/etc/syslog-ng/syslog-ng.conf  --name syslog-ng   -p 514:514 -p 601:601   balabit/syslog-ng:latest

docker: Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/demo/syslog-ng/syslog-ng.conf\\\" to rootfs \\\"/var/lib/docker/devicemapper/mnt/4751fd8ef1ddb782156a57749429486610b6c3d532a77c6ebf0727ca72ea451a/rootfs\\\" at \\\"/var/lib/docker/devicemapper/mnt/4751fd8ef1ddb782156a57749429486610b6c3d532a77c6ebf0727ca72ea451a/rootfs/etc/syslog-ng/syslog-ng.conf\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type.

root@hdata3:/home/smb/code/conf#

root@hdata3:/home/smb/code/conf# docker volume ls

DRIVER              VOLUME NAME

root@hdata3:/home/smb/code/conf#

On 25 May 2018 at 17:01, László Várady <laszlo.varady@balabit.com> wrote:

Hi Mujeeb,

Something weird happened when you executed

docker run -it -v "$PWD"/syslog-ng.conf:/etc/syslog-ng/syslog-ng.conf --name syslog-ng -p 514:514 -p 601:601 balabit/syslog-ng:latest

for the first time, so I would like to ask you to start from a clean environment.

- You've made sure that /home/smb/code/conf/syslog-ng.conf exists on your host and it is a file.

- In the original balabit/syslog-ng image, /etc/syslog-ng/syslog-ng.conf is also a file, so the mount should work.

Please run the following commands:

1. docker rm syslog-ng

2. docker rmi balabit/syslog-ng

3. docker pull balabit/syslog-ng

And finally, what you actually wanted to run:

4. docker run -it -v /home/smb/code/conf/syslog-ng.conf:/etc/syslog-ng/syslog-ng.conf --name syslog-ng -p 514:514 -p 601:601 balabit/syslog-ng:latest

If you still experience problems after this, please send us the output of "docker volume ls".

One more question:

Is it possible that "/home/smb/code/conf/" is a mounted network filesystem (NFS, SMB)?

--

László Várady

On Fri, May 25, 2018 at 5:33 PM Mujeeb Baig <baig.mujeeb@gmail.com> wrote:

Hi Laci,

It runs fine

root@hdata3:/home/smb/code/conf# file /home/smb/code/conf/syslog-ng.conf

/home/smb/code/conf/syslog-ng.conf: ASCII text

Regards

Mujeeb

On 25 May 2018 at 16:20, Szemere, László <laszlo.szemere@balabit.com> wrote:

Hello,

 Thank you for the output. Can you run the

file /home/smb/code/conf/syslog-ng.conf

 command on your HOST machine?

Br,

Laci

On Fri, May 25, 2018 at 5:04 PM, Mujeeb Baig <baig.mujeeb@gmail.com> wrote:

It is locating the source correctly, this is what I see under 'Mounts' section of inspect

        "Mounts": [

            {

                "Type": "bind",

                "Source": "/home/smb/code/conf/syslog-ng.conf",

                "Destination": "/1234",

                "Mode": "",

                "RW": true,

                "Propagation": "rprivate"

            }

        ],

On 25 May 2018 at 15:57, Szemere, László <laszlo.szemere@balabit.com> wrote:

Hello,

 From the Docker documentation: https://docs.docker.com/v17.09/engine/admin/volumes/bind-mounts/#differences-between--v-and---mount-behavior

If you use -v or --volume to bind-mount a file or directory that does not yet exist on the Docker host, -v will create the endpoint for you. It is always created as a directory.
If you use --mount to bind-mount a file or directory that does not yet exist on the Docker host, Docker does not automatically create it for you, but generates an error.

 It looks like docker daemon still can not find your file "$PWD"/syslog-ng.conf and creates a directory instead.

 Instead of heavy debugging, I suggest to run docker inspect syslog-ng to see what path was actually mounted by the docker daemon. You should see something like this in the output:

 "Mounts": [
            {
                "Type": "bind",
                "Source": "XXX/syslog-ng.conf",
                "Destination": "/1234",
                "Mode": "",
                "RW": true,
                "Propagation": "rprivate"
            }

 Once we have the actual mounted path, it will be easier to figure out why the docker daemon can not find your original config file.

Br,

Laci

On Fri, May 25, 2018 at 4:19 PM, Mujeeb Baig <baig.mujeeb@gmail.com> wrote:

Hi Andrew,

Thanks for your response. Yes the path is correct

When I execute with host location as /1234 it creates the container, but nothing inside /1234 folder

root@hdata3:/home/smb/code/conf# docker run -it -v "$PWD"/syslog-ng.conf:/1234  --name syslog-ng   -p 514:514 -p 601:601   balabit/syslog-ng:latest

syslog-ng: Error setting capabilities, capability management disabled; error='Operation not permitted'

[2018-05-25T14:12:15.695588] WARNING: Configuration file format is too old, syslog-ng is running in compatibility mode. Please update it to use the syslog-ng 3.14 format at your time of convenience. To upgrade the configuration, please review the warnings about incompatible changes printed by syslog-ng, and once completed change the @version header at the top of the configuration file.;

I can see inside the container that  /etc/syslog-ng/ this folder is not updated

root@hdata3:/home/smb# docker exec -ti syslog-ng /bin/bash

root@d634d32aee4c:/#

root@d634d32aee4c:/# ls -l /1234/

total 0

root@d634d32aee4c:/#

root@d634d32aee4c:/# ls -l /sy/etcslog-ng/

conf.d/         patterndb.d/    scl.conf        syslog-ng.conf

root@d634d32aee4c:/# ls -lrt /etc/syslog-ng/

total 12

-rw-r--r--. 1 root root 5910 Feb 23 13:17 syslog-ng.conf

drwxr-xr-x. 2 root root    6 Mar  2 12:58 conf.d

drwxr-xr-x. 2 root root    6 Mar  2 12:58 patterndb.d

-rw-r--r--. 1 root root 1336 Mar  2 12:58 scl.conf

root@d634d32aee4c:/#

On 25 May 2018 at 13:31, Mitzki, András <andras.mitzki@balabit.com> wrote:

Hi Mujeeb,

Maybe the problem is with the config file path:  "/home/smb/code/conf/syslog-ng.conf"

Could you check that path is correct?

Regards

Andrew

On Fri, May 25, 2018 at 1:13 PM, Mujeeb Baig <baig.mujeeb@gmail.com> wrote:

Hi

This is my first trial of syslog-ng. I've tried to start syslog-ng just like described on the official bolg page https://syslog-ng.com/blog/central-log-server-docker/ https://syslog-ng.com/blog/collecting-docker-infrastructure-logs-using-syslog-ng/

I am starting docker container as:

docker run -it -v "$PWD"/syslog-ng.conf:/etc/syslog-ng/syslog-ng.conf --name syslog-ng -p 514:514 -p 601:601 balabit/syslog-ng:latest

But unfortunately it is failing to mount my custom (local) configuration file from source to host, complaining about mounting a directory onto file

Error:

docker: Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\"/home/smb/code/conf/syslog-ng.conf\\" to rootfs \\"/var/lib/docker/devicemapper/mnt/52d32f854a030b396b03e7596ab5d71eb1a18f34a09d5e4997c437568749b259/rootfs\\" at \\"/var/lib/docker/devicemapper/mnt/52d32f854a030b396b03e7596ab5d71eb1a18f34a09d5e4997c437568749b259/rootfs/etc/syslog-ng/syslog-ng.conf\\" caused \\"not a directory\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type.

Am I missing something?

Please let me know

Many Thanks

Mujeeb

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq