Blank line in aws name

[Yotam_Hershkovitz]

Hi,

I'm a bit new to all of this so I hope I'll be clear enough.

We're seeing in logstash a blank line as seen in the picture.
Even when I export it in raw or formatted the blank line is there with the count.

What can be the issue?

Thanks

[Larry_Gregory]

Hi @Yotam_Hershkovitz,

How are you getting your data into Elasticsearch? It seems like you have a number of documents with either a missing or empty aws-name field, and the missing/empty field is treated like any other entry when performing aggregations like this.

[Yotam_Hershkovitz]

From what I understand we have linux servers with logstash installed on them and they send the data (sorry if this is obvious) to the kibana server.
I can't seem to figure out where does the aws server names comes from, I mean where does logstash gets the server names?
it's not from the server itself since there's nowhere the hostname appears.

it there a conf file with the name somewhere?

any suggestions ?

Im sorry if what I wrote isn't correct / clear / or really obvious

[Larry_Gregory]

Hi @Yotam_Hershkovitz, I've moved this to the Logstash forum, so they can help you with your Logstash configuration.

[magnusbaeck (Magnus Bäck)]

I can't seem to figure out where does the aws server names comes from, I mean where does logstash gets the server names?

That's entirely up to your Logstash configuration, so unless you show it to us we can't possibly help.

[Yotam_Hershkovitz]

Here's the logstash config file: /etc/logstash/logstash.conf

   output {
    if "ERROR" in [level]
    {
      datadog {
        api_key => xxx_I_prefer_not_writing it here_xxxxxx"
        alert_type => "error"
        title => "Logstash event for ${HOSTNAME:NA}, Instance-Id: ${AWS_ID:NA}, Name-Tag: ${AWS_NAME:NA}"
      }
    }
    elasticsearch {
        hosts => ['IP1:PORT', 'IP2:PORT']
        index => 'logstash-%{+YYYY.MM.dd}'
    }
}

input {
  file {
    path => '/workspace/server-name-*/logs/*.log'
    type => 'erver-name'
    codec => multiline {
      pattern => '^%{TIMESTAMP_ISO8601} '
      negate => true
      what => previous
    }
  }
}

filter {
  mutate {
    remove_field => [ "host", "aws-id", "host-ip", "aws-name" ]
  }
  mutate {
    add_field => {
      "host" => "${HOSTNAME:NA}"
      "aws-id" => "${AWS_ID:NA}"
      "host-ip" => "${HOST_IP:NA}"
      "aws-name" => "${AWS_NAME:NA}"
    }
  }
  grok {
    match => { 'message' => '%{TIMESTAMP_ISO8601:timestamp}%{SPACE}%{LOGLEVEL:level}%{SPACE}\[%{DATA:thread}]%{SPACE}%{USERNAME:file}~%{USERNAME:method}%{SPACE}%{GREEDYDATA:logmsg}' }
  }
  date {
    match => [ 'timestamp', 'yyyy-MM-dd HH:mm:ss,SSS' ]

[magnusbaeck (Magnus Bäck)]

According to your configuration the aws-name field comes from the AWS_NAME environment variable.