Last empty field

[Emanuel_Prado]

How can I get it to be ingested when the last information field is empty?

Example all fields complete:
05.03.2021 06:20:00;BTUC8;10168.00;66.66;66.39;0.00;1.60
05.03.2021 06:20:00;BTUC8;10168.00;66.66;66.39;0.00;1.60
05.03.2021 06:20:00;BTUC8;10168.00;66.66;66.39;0.00;1.60

Example last field empty:
05.03.2021 06:20:00;BTUC8;10168.00;66.66;66.39;0.00;
05.03.2021 06:20:00;BTUC8;10168.00;66.66;66.39;0.00;
05.03.2021 06:20:00;BTUC8;10168.00;66.66;66.39;0.00;

I already tried this but it doesn't work (It only works when there is value in the last field):

{
"set": {
"field": "LastField",
"value": 0.0
}
},

Thanks!

[spinscale (Alexander Reelsen)]

Hey,

is it possible that you provide a fully reproducible example with more context? otherwise it will be really hard to help.

Thank you!

--Alex

[Emanuel_Prado]

Hi Alex, here more info:

FILE .conf

input {
file {
type => "Prueba"
path => "/XX/BSC*.csv"
start_position => "beginning"
ignore_older => 360000
max_open_files => 10000
}
}

output {
if "Prueba" == [type] {
elasticsearch {
index => "index"
hosts => ["https://xx.xx.xx.xx:9200"]
pipeline => "BSC"
ssl => true
ssl_certificate_verification => false
manage_template => false
}
}
}

BSC pipeline

PUT _ingest/pipeline/BSC
{
"description": "BSC",
"processors": [
{
"csv": {
"field": "message",
"target_fields": [
"TIMESTAMP",
"Field 2",
"Field 3",
"Field 4",
"Field 5",
"Field 6",
"Last Field"
],
"separator": ";"
}
},

{
  "date": {
    "field": "TIMESTAMP",
    "target_field": "@timestamp",
    "formats": [
      "MM.dd.YYYY HH:mm:ss"
    ],
    "timezone": "America/Argentina/Cordoba"
  }
},
{
  "remove": {
    "field": [
      "TIMESTAMP",
      "message",
      "host",
      "path"
    ]
  }
}

]
}

Thanks!!!!