Logstash has fetched the data but not shipped into Elasticsearch no error in logstash cmd

Elastic Stack Logstash
1

Hi,

I am fetching data from another database server.
data is fetching but not inserted into elastcisearch,
No error is showing in logstash cmd

2

Hi @swati_dupare
Thank you for reaching out. I've moved you topic from kibana to logstash, since it's logstash related, could you add some details, e.g. your logstash pipeline configuration, this would help debugging your problem
Best,
Matthias

3

input {
jdbc {
jdbc_driver_class => "com.mysql.cj.jdbc.Driver"
jdbc_connection_string => "jdbc:mysql://remote-ip-address:3306/testdb?serverTimezone=Asia/Kolkata"
jdbc_user => "root"
jdbc_password => "xxxx"
tracking_column => "unix_ts_in_secs"
use_column_value=>true
statement => "select * from user"
schedule => "0 */15 * * * *"
}
}

output {
elasticsearch {
document_type => "doc"
document_id=> "%{id}"
index => "testindex"
hosts => ["http://localhost:9200"]
}
stdout{

codec => rubydebug
}
}

4

so it's written to stdout, but not to Elasticsearch?

5

Means..?
Actually,If connect to localhost , it's working fine ,but when i trying to connect it to remote database server data is not pushed to elasticsearch

6

You're receiving the data from localhost successfully, so it's written into Elasticsearch. But it doesn't work when the database is remote? thx!

7

so, What I need to do.??

8

You need to make sure that logstash can connect to that remote db server, first you could check it it's possible to connect to the db instance without logstash.

9

Actually,when I run my logstash configuration file "select * from user" statement is executed and it shows data from remote server database in logstash cmd,but not pushing into elasticsearch

10

so i misunderstood this, it doesn't work when Elasticsearch is remote, just when local?

11

yes.. when connected to local, it's works properly.

12

then you should try if you can connect to your remote ES machine without logstash (e.g. with curl )

13

my logstash and elasticsearch are on same server only database which I am trying to connect is on another server

14

So the data is displayed in logstash (how?) but not stored in elasticsearch? could you try to remove document_id=> "%{id}" to see if it works in this case?
thx

15

Yeah... It's working now.. issue with duplicate document ID.
Thanx

1 Like
16

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.