Is it possible to write to /dev/stdout using file output?


(Ks Chan) #1

I was trying to use /dev/stdout as the beats output but failed to do so.

The config is like ...

### File as output
  file:
    # Enabling file output.
    enabled: true

    # Path to the directory where to save the generated files. The option is mandatory.
    path: "/dev"

    # Name of the generated files. The default is `topbeat` and it generates...
    filename: stdout

BTW, I just hope to use /dev/stdout for easier debugging of the beats output.


(ruflin) #2

@mrkschan Interesting idea. I actually never tried that. This will probably fail as the file output tries to create and rotate files which I don't think is possible for /dev/stdout. I would recommend to create a local file and tail it for debugging at the moment.

What is the error you get with the above?


(Ks Chan) #3

No error can be spotted actually... The beats output just disappear and I don't even get any info from -d '*' debug selector. @ruflin


(Monica Sarbu) #4

Yes, indeed. I get the same behavior. No warning is printed out. I created a bug in libbeat and it's targeted to be fixed for the next release 1.0.0-GA: https://github.com/elastic/libbeat/issues/212


(Steffen Siering) #5

@mrkschan is -d 'publish' ok for your purpose? I once used to create a stdout logger myself, but found -d 'publish' as well as good.


(Ks Chan) #6

@steffens that is great! Though, why -d '*' cannot catch this or do I miss something?

BTW, I'm thinking if stdout is an viable output option in libbeat. Consider the following cases:

  1. libbeat is used in a custom beat that is required to send events offline (e.g. latency requirement, durability requirement, etc.), either file output or stdout will be used and later ship via filebeat for instance. If that beat is executed inside a Docker container, stdout would be a preferable option.
  2. libbeat is used in a custom beat that is required to send outputs to stdout according to the ops team so that it can be integrated with syslog easily (there's some good reason behind this - https://hynek.me/articles/taking-some-pain-out-of-python-logging/).
  3. straight forward debug / development option.

Though I know that I make up these cases, libbeat as a library may not prevent a common output option - stdout.


(Monica Sarbu) #7

I created a ticket in GitHub to add /dev/stdout as file output: https://github.com/elastic/libbeat/issues/218


(Steffen Siering) #8

we found a bug in -d '' : https://github.com/elastic/libbeat/pull/226
-d '
' should work in next releases


(system) #9