Missing headers even with include_headers (Csv codec plugin) set to true

[alexus]

Hello World!

per Csv codec plugin | Logstash Reference [7.17] | Elastic

I'm set include_headers flag to value true, yet even though headers gets included into output on first run, at later time on re-run schedule of very same pipeline, the headers are no longer included into output...

per the above mention documentation, may be it has something to do with this?

When encoding in an output plugin, include headers in the encoded CSV once per codec lifecyle (not for every event)

---

How do I enforce include_headers at ALL time (not the just first run)?

Please advise)
Thank you in advance!

[Badger]

@alexus

at later time on re-run schedule of very same pipeline

It is unclear what you mean by that. How are you scheduling it?

[alexus]

@Badger thank you for your reply.

I use scheduling for schedule of re-run of same pipeline.

[Badger]

@alexus

I use scheduling for schedule of re-run of same pipeline.

The codec only outputs headers once in its lifecycle. With scheduling the pipeline does not restart, it just sleeps until the next scheduled time, so it is not going to output headers again.

[alexus]

ok, thanks for the explanation)

a follow up question, is there a way to enforce it every time (vs first time only) somehow?

[Badger]

@alexus

is there a way to enforce it every time (vs first time only)

Not within logstash itself. It could probably be done externally. For example, if you have --config.reload.automatic enabled and path.config pointed to a directory and you want to schedule execution every 10 minutes, then you could create a file with a single blank line in the directory at 10, 30, and 50 past the hour, and remove that file at 00, 20, and 40 past the hour. That would trigger a reload of the pipeline which would cause the headers to be written again. You would then remove the scheduling from the input.