Hello,
I am very new to ELK , need your guidance in my below usecase development.
We have lot of XML files generated from source server(around 55GB per day )..but, from each file, we need only very few lines, as shown below.
** **
** **
** **
....(248 lines)
** **
** <TX_COMMIT timestamp="1574339699585" so="103"/>**
We need only 1) name 2) timestamp(command,tx_commit) 3) val="xxxxxx" 4) SO_ID
To avoid , transferring all the files clogging bandwidth & filesize generated, I am trying to discard tag inside , from the source itself.
I tried using in filebeat.yml, but i am not sure if this the best approach.
exclude_lines: ['(\n)*']
i know we can use multiline.pattern but dont know how to exactly discard the above pattern.
Any support or guidance will be much appreciated...