[SOLVE] Logstash output works but fail to feed elasticsearch

Elastic Stack Logstash
1

I am trying to pass my API custom log files to Kibana. For that I use myLogFiles.log -> Logstash -> Elasticsearch -> Kibana.

PROBLEM

Logstash output tells that everything is fine, but Elasticsearch remain empty (I checked that with kibana or elasticsearch web visualisation plugins)

QUESTION

How can I feed elasticsearch to visualise my data in kibana ?

DEBUG SO FAR

To have some debug I told Logstash to ouput in a .log file and on console. see my logstash.conf:

input {
    file {
        path => '/home/***/dev_logstach/acci.log'
    start_position => beginning
    }
}


filter {
    grok {
        match => {"message" =>  "%{WORD:key} %{WORD:userID} %{WORD:lakeID} %{WORD:ballID} %{NUMBER:longitude} %{NUMBER:latitude} %{TIMESTAMP_ISO8601:date_evenement}"}
    }
}


output {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "testACCI"
  }
  stdout { codec => rubydebug }
  file {
   path => "/home/***/dev_logstach/testLog.log"
   create_if_deleted => true
  }
}

And a logstash output with my datas

{
            "userID" => "HJcOX",
        "@timestamp" => 2018-04-10T13:27:47.723Z,
          "latitude" => "-123.8",
           "message" => "{message:ACWWCI HJcOX qHYFM ABCCCC -22.5 -123.8 2018-04-10T09:11:06.173Z,level:info}",
              "path" => "/home/***/dev_logstach/acci.log",
    "date_evenement" => "2018-04-10T09:11:06.173Z",
          "@version" => "1",
            "lakeID" => "qHYFM",
               "key" => "ACWWCI",
         "ballID" => "ABCCCC",
              "host" => "sd-10****",
         "longitude" => "-22.5"
}

When I run again it to not log (or print on console) previous data, and when logstash run if I

echo -e "new data line" >> acci.log
the new data appear in the logs. I then suppose that the data are send somewhere but I have no clue where and how can I move forward.

In elasticsearch log I just see a warning:

2018-04-10T17:01:52,507][WARN ][o.e.d.i.m.MapperService  ] [_default_] mapping is deprecated since it is not useful anymore now that indexes cannot have more than one type

P.S. I read some problem like that but it was due to date format mine is valide (YYYY-MM-DD...)

2

I believe index names must be lowercase, which might be causing issues. I would expect you to see this in the logs if you enable debug logging.

3

[FIX]

Well I had 2 errors, the first, as you said and the second one, I had to first create the index in elasticsearch.

Thanks

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.