# MySQL fields not visible in Kibana

**URL:** <https://discuss.elastic.co/t/mysql-fields-not-visible-in-kibana/82574>\
**Category:** Logstash\
**Created:** [April 17, 2017, 12:35pm UTC](https://discuss.elastic.co/t/mysql-fields-not-visible-in-kibana/82574 "2017-04-17T12:35:32Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ferdi](https://avatars.discourse-cdn.com/v4/letter/f/e19adc/32.png) [@Ferdi](https://discuss.elastic.co/u/Ferdi)\
**Post date:** [April 17, 2017, 12:35pm UTC](https://discuss.elastic.co/t/mysql-fields-not-visible-in-kibana/82574/1 "2017-04-17T12:35:32Z")

</div>

I have loaded data into Elastic Seach using logstash and a JDBC MysQL driver.

However, in Kibana, I see stuff like:

cluster\_uuid:xxxxxxxxxxxxxxxxx timestamp:April 11th 2017, 05:37:39.580 source\_node.uuid:yyyyyyyyyyyyyyyysource\_node.host:12.18.169.0 source\_node.transport\_address:13.0.18.169:12346 source\_node.ip:10.18.169.0 source\_node.name:instance-0000000000 source\_node.attributes.logical\_availability\_zone:zone-0 source\_node.attributes.availability\_zone:us-west-4d-2048source\_node.attributes.region:us-west-2 kibana\_stats.kibana.uuid:uuid-xxxxx-yyyyy-zzzzzzz

... and I cannot see any of the fields from my MySQL table.  
I get the feeling I'm missing something obvious.

What is the simplest config that I can use to load MySQL data and configure things like count, average and sum for different columns?

---

<div class="post-metadata">

**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\
**Post date:** [April 17, 2017, 7:47pm UTC](https://discuss.elastic.co/t/mysql-fields-not-visible-in-kibana/82574/2 "2017-04-17T19:47:42Z")

</div>

Hi Ferdi,

Based on the stuff you are seeing in Kibana, it _seems_ to me that you might be pointing to a different index in Kibana than the one containing your data from MySQL.

What is the name of the index (or indices) containing your MySQL data? If you didn't customize it in your Logstash config's `elasticsearch` output section, the name will be `logstash -<YYYY.MM.DD>` (that is, one index per date).

---

<div class="post-metadata">

**Author:** ![Ferdi](https://avatars.discourse-cdn.com/v4/letter/f/e19adc/32.png) [@Ferdi](https://discuss.elastic.co/u/Ferdi)\
**Post date:** [April 18, 2017, 1:41pm UTC](https://discuss.elastic.co/t/mysql-fields-not-visible-in-kibana/82574/3 "2017-04-18T13:41:04Z")

</div>

Thank you for your reply @shaunak  
This is my logstash config:

```
input {
        jdbc {
                jdbc_connection_string => "jdbc:mysql://localhost:3306/pone_test"
                jdbc_user => "mysql_user"
                jdbc_password => "mysql_password"
                jdbc_driver_library => "/usr/lib/jvm/java-8-oracle/jre/lib/mysql-connector-java-5.1.41-bin.jar"
                jdbc_driver_class => "com.mysql.jdbc.Driver"
                statement => "SELECT * FROM estimate WHERE bill_no > 0"
                add_field => {
                        "field1" => "estimate_id"
                        "field2" => "estimate_uuid"
                        "field3" => "order_no"
                        "field4" => "order_dt"
                        "field5" => "bill_dt"
                        "field6" => "bill_code"
                        "field7" => "net_amount"
                }
        }
}

output {
        stdout { codec => json_lines }
        elasticsearch {
                hosts => "https://elastic_endpoint:9243/"
                user => "elastic_user"
                password => "elastic_password"
                index => "pone_test"
        }

}

```

In the Kibana management, index pattern is filled in by default to logstash-\*.  
If I fill in pone\_test, I get the message 'Unable to fetch mapping. Do you have indices matching the pattern?' and am unable to save the setting. So I used \*.

I'm still confused and going around in circles. I just want to see my mysql table fields in Kibana. All I currently see is all sorts of json data with IPs and cluster IDs and what not, but none of my mysql fields.

---

<div class="post-metadata">

**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\
**Post date:** [April 18, 2017, 1:53pm UTC](https://discuss.elastic.co/t/mysql-fields-not-visible-in-kibana/82574/4 "2017-04-18T13:53:42Z")

</div>

If you are getting the 'Unable to fetch mapping. Do you have indices matching the pattern?' error in Kibana, its quite possible the `pone_test` index hasn't been created in Elasticsearch. What does the output of https://elastic\_endpoint:9243/\_cat/indices/pone\* show you?

---

<div class="post-metadata">

**Author:** ![Ferdi](https://avatars.discourse-cdn.com/v4/letter/f/e19adc/32.png) [@Ferdi](https://discuss.elastic.co/u/Ferdi)\
**Post date:** [April 18, 2017, 3:21pm UTC](https://discuss.elastic.co/t/mysql-fields-not-visible-in-kibana/82574/5 "2017-04-18T15:21:54Z")

</div>

@shaunak

\_cat/indices/pone\* is blank

\_cat/indices/\* returns:  
yellow open .kibana UIAlyHvJSVG7nVKgtgH-pw 1 1 2 0 13.5kb 13.5kb  
yellow open .monitoring-kibana-2-2017.04.18 p1AgpC6aT0qEF9Ws0sUmBw 1 1 1000 0 538.3kb 538.3kb  
yellow open .monitoring-data-2 XLlJSvmOQ\_ixIaXeagVglw 1 1 1 0 5.9kb 5.9kb  
green open .security dUp7IUwnQO2bNPND8j8rKw 1 0

The closest thing I found to setting an index in ElasticSearch is Automatic index creation enable/disable.

---

<div class="post-metadata">

**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\
**Post date:** [April 18, 2017, 3:40pm UTC](https://discuss.elastic.co/t/mysql-fields-not-visible-in-kibana/82574/6 "2017-04-18T15:40:47Z")

</div>

So it looks like the `pone_test` index isn't even being created. When you run Logstash with that config, do you see any errors? Also, since you are using the `stdout` output plugin as well, do you see any of your MySQL data being output to STDOUT by Logstash?

---

<div class="post-metadata">

**Author:** ![Ferdi](https://avatars.discourse-cdn.com/v4/letter/f/e19adc/32.png) [@Ferdi](https://discuss.elastic.co/u/Ferdi)\
**Post date:** [April 18, 2017, 5:54pm UTC](https://discuss.elastic.co/t/mysql-fields-not-visible-in-kibana/82574/7 "2017-04-18T17:54:14Z")

</div>

Thanks for your patience @shaunak  
Ok, looks like now we'll get somewhere 🙂

I ran logstash again and this was the output:

```
$ /opt/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf
Settings: Default pipeline workers: 4
Exception when executing JDBC query {:exception=>#<Sequel::DatabaseError: Java::JavaSql::SQLException: Value '0000-00-00 00:00:00' can not be represented as java.sql.Timestamp>, :level=>:warn}
Logstash startup completed
Logstash shutdown completed

```

I assumed this was a warning that only affected records with the 0000-00-00 00:00:00 date.  
Now what do I do?

---

<div class="post-metadata">

**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\
**Post date:** [April 18, 2017, 6:06pm UTC](https://discuss.elastic.co/t/mysql-fields-not-visible-in-kibana/82574/8 "2017-04-18T18:06:24Z")

</div>

Hi Ferdi,

I don't know enough about the JDBC input plugin to help, but someone in the Logstash category might. I'll move this post over there now.

Shaunak

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 16, 2017, 6:14pm UTC](https://discuss.elastic.co/t/mysql-fields-not-visible-in-kibana/82574/9 "2017-05-16T18:14:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
