Ingest (from filebeat): How to include date from log file header in each log line?

[sockit]

I am using filebeat to send data directly to an elasticsearch ingest pipeline.
The (daily) log files each have a date for the file in a header line at the beginning of the file:

` ** Application log ** Current day: 02/18/19 **`

(18:44:30.379)(04332)Message_type_1: Descriptive text of log message
(18:44:30.399)(04333)Message_type_2: Log msg received (001) 02 30 31 36 37 34 30 38 31 36 33 30 32 30 37 38 38 35 30 35

When the ingest pipeline is configured as below, I get the msg_time (as expected) as:

`"msg_time" : "23:56:18.974"`

I would like to have each msg_time value include the log file date, like:

`"msg_time" : "2019-02-18 23:56:18.974"`

How can this be done (preferably without using logstash as an intermediary)?

Thanks in advance.

======
Ingest Pipeline definition:
#!/bin/sh
curl -XPUT "http://localhost:9200/_ingest/pipeline/test" -H 'Content-Type: application/json' -d'
{
"description" : "PUT _ingest/pipeline/test: Convert test log data to indexed data",
"version" : "1",
"processors" : [
{ "grok": {
"field": "message",
"patterns": [ "\(%{TIME:msg_time}\)\(%{NUMBER:seqnum}\)%{DATA:msg_type}:%{GREEDYDATA:msg_text}" ]
}
},
{
"convert": {
"field" : "seqnum",
"type": "integer"
}
}
],
"on_failure" : [
{
"set" : {
"field" : "error",
"value" : "{{ _ingest.on_failure_message }}"
}
}
]
}'

[dadoonet (David Pilato)]

I don't think you can.

[sockit]

Thanks for the response. Do you mean :

1. it can't be done at all?
2. it can't be done without using logstash? (If so, how to use logstash for this?)

[dadoonet (David Pilato)]

Even with logstash it might be hard to do this.
Because each line is basically a new document without a real "context".

My best guess is to extract the first line "manually" (script shell may be) and start filebeat to ingest that file (skip the first line) and add as a variable may be the content that you extracted manually.

I don't know if it's doable but you could ask in #beats:filebeat for help.