Index not created using log stash

Elastic Stack Logstash
,
1

I am using elasticsearch with logstash [Version : 7.9.0] to visualize a data set with kibana.

I am doing this task on Windows10.

Logstash config file -

input {
  jdbc {
    jdbc_driver_library => "$PATH\mysql-connector-java-5.1.46.jar"
    jdbc_driver_class => "com.mysql.jdbc.Driver"
    jdbc_connection_string => "jdbc:mysql://localhost:3306/world?autoReconnect=true&useSSL=false"
    jdbc_user => root
    jdbc_password => herculus
    parameters => { "names" => "Kabul" }
    statement => "SELECT count(*) FROM world.city where name = :names;"
    schedule => " * * * * * *"
  }
}
output {
  elasticsearch {
    document_id => "%{id}"
    document_type => city
    index => cities
    hosts => ["http://localhost:9200"]
  }
  stdout{
  codec => rubydebug
  }
}

Response to curl command,

logstash-7.9.0\bin>curl -X GET "localhost:9200/cities/_search"
{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index [cities]","resource.type":"index_or_alias","resource.id":"cities","index_uuid":"_na_","index":"cities"}],"type":"index_not_found_exception","reason":"no such index [cities]","resource.type":"index_or_alias","resource.id":"cities","index_uuid":"_na_","index":"cities"},"status":404}

Also, for Get request to URL : http://localhost:9200/_cat/indices?v&pretty

O/P -

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size

When I updated the logstash.conf to ,

input {
  jdbc {
    jdbc_driver_library => "$PATH\mysql-connector-java-5.1.46.jar"
    jdbc_driver_class => "com.mysql.jdbc.Driver"
    jdbc_connection_string => "jdbc:mysql://localhost:3306/world?autoReconnect=true&useSSL=false"
    jdbc_user => root
    jdbc_password => herculus
    parameters => { "names" => "Kabul" }
    statement => "SELECT * FROM world.city where name = :names;"
    schedule => " * * * * * *"
  }
}
output {
if "world.city" in [tags] {
  elasticsearch {
    # document_id => "%{id}"
    # document_type => city
    index => "cities"
    hosts => ["http://localhost:9200"]
  }
  }
  stdout{
  codec => rubydebug
  }

}

I am getting response

[2020-08-26T10:28:01,229][INFO ][logstash.inputs.jdbc     ][main][3b01a5ef336ad85c3946fe6866ca42078069f817e05ec92d17e8ab77baa85416] (0.002075s) SELECT * FROM world.city where name = 'Kabul';
{
     "population" => 100000,
       "@version" => "1",
     "@timestamp" => 2020-08-26T04:58:01.231Z,
           "name" => "Kabul",
    "countrycode" => "IDN",
       "district" => "Yasjdasjk",
             "id" => 4081
}
{
     "population" => 10000,
       "@version" => "1",
     "@timestamp" => 2020-08-26T04:58:01.231Z,
           "name" => "Kabul",
    "countrycode" => "IDN",
       "district" => "Uhjhgjh",
             "id" => 4082
}
2

You need to have quotes around the index value. You should not be using type though.

3

Why do you expect it to have that value in the [tags] array?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.