How can I import json data from Mysql 5.5 to logstash 1.4.2?

Elastic Stack Logstash
1

Hi,

I am using ELK stack on an Ubuntu 14.04 server. The versions of each application is as follows:
Elasticsearch: 0.90.13
Logstash: 1.4.2
Kibana: 3.0.1

I have succeeded in establishing connection between the three applications and logs are being displayed in kibana.

I have been asked to log some of the json data that was being stored in mysql to centralised logging system as well.

I have tried jdbc-river but its been futile, since I can't get it to import my logs from mysql.

I cannot upgrade elasticsearch as several components in my project are not compatible with higher versions and it will create conflict.

Please help!

PS: I am not getting any logs registered in /var/log/elasticsearch/elasticsearch.log file.Althought there should have been loads of logs.

2

You should really upgrade, those are very, very old versions. At least upgrade Logstash, it's not locked to any ES version.

Have you seen https://www.elastic.co/guide/en/logstash/current/plugins-inputs-jdbc.html?

1 Like
3

Thank you so much. I was not upgrading logstash because I thought it was same as kibana and ES. That it needed to be of specific version. Thank you for all the help.

4

@pythonl1 can you plz help to set up the connection btw three applications?

i/m not able to seee logs in kibana?

input {
tcp {
port => 5000
type => "company-tcp"
codec => "json"
}
udp {
port => 5000
type => "company-udp"
codec => "json"
}
}

output {
if ("METRICS" in [tags]) {
elasticsearch {
hosts => ["elasticsearch"]
index => "metrics-%{+YYYY.MM.dd}"
}
} else {
elasticsearch {
hosts => ["elasticsearch"]
index => "logstash-%{+YYYY.MM.dd}"
}
}
stdout {
codec => rubydebug
}
}

5

@Lakshmi_D I think your code should be something like this:

 {
	tcp {
		hosts => [ "ip_from_where_your_data_is_being_called:5000" ]
		port => 5000
		type => "company-tcp"
		codec => "json"
	}
	udp {
		hosts => [ "ip_from_where_your_data_is_being_called:5000" ]
		port => 5000
		type => "company-udp"
		codec => "json"
	}
}

output {
	if [tag] == "metrics" {
		elasticsearch {
			hosts => ["elasticsearch_ip_or_dns"]
			index => "metrics-%{+YYYY.MM.dd}"
		}
	} else {
		elasticsearch {
			hosts => ["elasticsearch_ip_or_dns"]
			index => "logstash-%{+YYYY.MM.dd}"
		}
	}
	stdout {
		codec => rubydebug
    }
}
6

Please start your own thread.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.