Some imported fields showing in discover but not in indices


(Thomas Wollschlager) #1

Hi Everyone,

I'm pretty new to ELK so maybe im missing something simple. When I import data from our Microsoft Sql DB all fields that are auto-generated by our DB (they all contain preceding double underscores) show up in the discover tab with a "?" to the left of it. Additionally they do not show up with the other fields and I have absolutely no interaction with them.

Here is a photo of the said fields in the discover tab.

As you can see from the bottom picture they do not show up as indices.

I'm extremely confused why Kibana or Logstash doesn't like these fields. I'm not sure if it's an issue with the naming convention (double underscore "__") or if its an issue that they were auto-generated by our DB. Addtionally i'm curious if there is some formatting I could do within Kibana that would allow me to use this data?

Thanks for any help,
Thomas Wollschlager


(Mark Walkom) #2

Huh?

Anyway, are you creating a mapping or using auto dynamic creation?


(Thomas Wollschlager) #3

Yeah sorry "indices" was probably not the best term for them. I basically just wanted to say I have no interaction with them aside from being able to simply view them in the discover tab.

I'm using dynamic creation. I've been reading for some data that's used dynamic creation will not be typed correctly so I will need to map them manually beforehand? or is there something else I can do to fix these specific types of data?


(system) #4