Filebeat cutting events from tiem to time

Elastic Stack Beats
1

Hi,
I have filebeat 7.3.0-1, that reads files from one directory and sends information to logstash, it process it and writes to elasticsearch.

I have a monitoring daemon that from time to time, gets a report from a database and writes a file on the directory logstash reads, the report might have several lines, in my case for the moment just one line, and writes line by line to the file.

/home | 25358| 30620|*ERROR*

every field is separated by a "|" and there might be several spaces before or after ther pipe and the value.

Filebeat reads line by line, filebeat.yml config is:

- type: log
  enabled: true
  paths:
    - /stage/logs/*chk_df_autoextend.txt
  fields:
    type: oracle
  fields_under_root: true
  harvester_buffer_size: 2097153
  close_inactive: 5m
  close_removed: true
  close_eof: true

then with logstash i process the information with the following filter:

filter {
 if [type] == "oracle" and [type2] == "df_autoextend" {
       #mount_point|free_mb|incremento|status
       dissect {
                mapping => {
                            "message" => "%{[df_autoextend][mount_point]->}|%{[df_autoextend][free_mb]}|%{[df_autoextend][incremento]}|%{[df_autoextend][status]->}"
                           }
               }
       mutate {
                convert => {
                  "[df_autoextend][mount_point]" => "string"
                  "[df_autoextend][free_mb]" => "integer"
                  "[df_autoextend][incremento]" => "integer"
                  "[df_autoextend][status]" => "string"
                }
              }
 }
}

Once in a while (once a day or even once with more than one day between) i get the message cut by logstash, for instance:

{
  "_index": "orachecks-2019.12",
  "_type": "_doc",
  "_id": "CSfE-W4BByjEVZh5ZWFm",
  "_version": 1,
  "_score": null,
  "_source": {
    "@timestamp": "2019-12-12T11:00:17.000Z",
    "type": "oracle",
    "ecs": {
      "version": "1.0.1"
    },
    "agent": {
      "ephemeral_id": "36696bd3-a1c0-4146-8cd2-8c0e5810a5cb",
      "version": "7.3.0",
      "hostname": "elk-lab01.tilsor.com.uy",
      "id": "7502e50a-1256-42de-b007-d0c5336841ce",
      "type": "filebeat"
    },
    "host": {
      "name": "elk-lab01.tilsor.com.uy"
    },
    "log": {
      "offset": 59,
      "file": {
        "path": "/stage/logs/2019-12-12-080017-chk_df_autoextend.txt"
      }
    },
    "type2": "df_autoextend",
    "tags": [],
    "df_autoextend": {
      "incremento": 30620,
      "mount_point": "                     ",
      "status": "*ERROR* ",
      "free_mb": 25358
    },
    "input": {
      "type": "log"
    },
    "@version": "1"
  },
  "fields": {
    "@timestamp": [
      "2019-12-12T11:00:17.000Z"
    ],
    "tbs_space.tbs_used_space": [
      null
    ],
    "tbs_space.tbs_status": [
      null
    ],
    "detail_diskgroups.dg_used_mb": [
      null
    ]
  },
  "highlight": {
    "type2.keyword": [
      "@kibana-highlighted-field@df_autoextend@/kibana-highlighted-field@"
    ]
  },
  "sort": [
    1576148417000
  ]
}

The problem is why do i get a message that start at offset 59 when it should read the whole line and send it to logstash??. Its causing logstash to generate incomplete information in elasticsearch. I could add some filter and discard the malformed events, but its not malformed in the original file.

Any clue?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.