Hi there,
I am new to ELK, I first tried to migrate data from MySQL to Elasticsearch using python using the tutorial in the below link. It worked.
Later, I tried to do it with Logstash, the data is migrated, but my PHP code is running into an error (displayed in the browser) as shown below.
Fatal error : Uncaught Elasticsearch\Common\Exceptions\UnexpectedValueException: "id" is not a valid parameter. Allowed parameters are "_source", "_source_exclude", "_source_include", "allow_no_indices", "analyze_wildcard", "analyzer", "batched_reduce_size", "client", "custom", "default_operator", "df", "docvalue_fields", "expand_wildcards", "explain", "fielddata_fields", "filter_path", "filter_path", "from", "human", "ignore_unavailable", "indices_boost", "lenient", "lowercase_expanded_terms", "preference", "q", "query_cache", "request_cache", "routing", "scroll", "search_type", "size", "slice", "sort", "source", "stats", "stored_fields", "suggest_field", "suggest_mode", "suggest_size", "suggest_text", "terminate_after", "timeout", "typed_keys", "version" in /var/www/html/elastic_php/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Endpoints/AbstractEndpoint.php:237 Stack trace: #0 /var/www/html/elastic_php/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Endpoints/AbstractEndpoint.php(74): Elasticsearch\Endpoints in /var/www/html/elastic_php/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Endpoints/AbstractEndpoint.php on line 237
Below is the Logstash.config file.
input {
jdbc {
id => "stateQuality"
jdbc_connection_string => "jdbc:mysql://localhost/telecom1"
# The user we wish to execute our statement as
jdbc_user => "xxxxx"
jdbc_password => "xxxxxxx"
# The path to our downloaded jdbc driver
jdbc_driver_library => "/usr/share/java/mysql-connector-java-8.0.16.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
# our query
statement => "SELECT * FROM state_quality"
}
}
output {
stdout { codec => json_lines }
elasticsearch {
hosts => "localhost:9200"
index => "state_index"
id => "%{id}"
}
}
I tried with "document_id" in the output part of the file, it didn't work
I tried with "_id" it didn't work.
I tried with just "id" it still didn't work.
Below is my PHP code
<?php //use Elasticsearch\ClientBuilder; require 'vendor/autoload.php'; $client = Elasticsearch\ClientBuilder::create()->build(); // if ($client) { // echo 'connected'; // } $params = [ 'index' => 'state_index', 'type' => '_doc', 'id' => 'new_user', ]; $response = $client->get($params); $response = $client->search($params); $hits = count($response['hits']['hits']); echo $hits; The tutorial I have linked to says the _id some kind of unique id for elasticsearch, I am not sure exactly what it is and how to set that in the logstash.config file and how to address that in PHP. Kindly provide some help.