Index created with wrong name

Hi,

I have a server with below setup

#cat /etc/redhat-release
CentOS Linux release 7.3.1611 (Core)

#timedatectl
      Local time: Thu 2017-01-12 07:21:08 GMT
  Universal time: Thu 2017-01-12 07:21:08 UTC
        RTC time: Thu 2017-01-12 07:21:08
       Time zone: Etc/GMT (GMT, +0000)
     NTP enabled: yes
NTP synchronized: yes
 RTC in local TZ: no
      DST active: n/a

#rpm -qa | grep logstash
logstash-5.1.1-1.noarch

#java -version
java version "1.8.0_112"
Java(TM) SE Runtime Environment (build 1.8.0_112-b15)
Java HotSpot(TM) 64-Bit Server VM (build 25.112-b15, mixed mode)

But for some reason the index being created is not what i specify in the config. Here is trace showing index created as powerdns-1-49002.07.31 even though i specify powerdns-1-%{+YYYY.MM.dd}. Can someone help explain why this is happening please ? To me the year is the last part of the hosts IP and port i.e. 4:9200 !

[2017-01-12T07:05:34,767][DEBUG][logstash.outputs.elasticsearch] config LogStash::Outputs::ElasticSearch/@hosts = ["10.149.62.14:9200"]
[2017-01-12T07:05:34,767][DEBUG][logstash.outputs.elasticsearch] config LogStash::Outputs::ElasticSearch/@index = "powerdns-1-%{+YYYY.MM.dd}"

....

[2017-01-12T07:06:31,501][DEBUG][org.apache.http.wire     ] http-outgoing-0 << "{"took":760,"errors":false,"items":[{"create":{"_index":"powerdns-1-49002.07.30","_type":"logs","_id":"CQcj3m4VRk2GCfW3fhxATQ","_version":1,"status":201}},{"create":{"_index":"powerdns-1-49002.07.31","_type":"logs","_id":"iTtXt6RaSAWz4afOVRsegA","_version":1,"status":201}},{"create":{"_index":"powerdns-1-49002.07.31","_type":"logs","_id":"RQJQfPuuR6i00vgPnNd-Hg","_version":1,"status":201}}]}"

Here is config i'm using

input {
  file {
    path => "/var/log/mariadb/mariadb.log"
    type => "mysql_log"
    start_position => "end"
  }
}

filter {
    if [type] == "mysql_log" {
    grok {
        patterns_dir => ["/etc/logstash/patterns"]
        match => { "message" => "%{MDBLOG}" }
    }

    mutate { rename => [ "message", "raw" ] }
    }
}

output {

  if "_grokparsefailure" in [tags] {
    file { path => "/var/log/logstash/failed_mariadb.log.events-%{+YYYY-MM-dd}.log" }
  }

  file {
    path => '/var/log/logstash/mariadb_log.json'
  }
  
  elasticsearch {
    hosts => ["10.149.62.14:9200"]
    index => "powerdns-1-%{+YYYY.MM.dd}"
  }
  
}

Does your message match?
What does that grok pattern look like? What does the data look like?

Yes the grok filter works fine.. but whats the grok pattern got to do with the index that logstash sends data too.. given that i explicitly tell it what it should be named.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.