Json import using data visualizer , json array shows values comma seperates

Elastic Stack Kibana
1

Kibana version : 7.13

Elasticsearch version : 7.13

APM Server version : 7.13

APM Agent language and version : java 1.8

Fresh install or upgraded from other version? fresh install

Is there anything special in your setup? For example, are you using the Logstash or Kafka outputs? Are you using a load balancer in front of the APM Servers? Have you changed index pattern, generated custom templates, changed agent configuration etc. :-no

I have imported a json data file using data visualizer. I have few fields with json array in it. SO after I import the data in discover the data within the array is shown as comma seperated values.

My data is displayed as

image
image800×141 8.3 KB

I am expecting it to be shown in discover as

BrandName:{
"Brand":"Optical"
},
{
"Brand":"Sun"
}

But instead it just displays as comma separated. Because of which when I apply a filter on dashboards to filter based on "Sun" brand it just filters based on both the "optical" and "sun" as same field has both the values.

Here is my mapping for that field

       "BrandName": {
          "properties": {
            "brand": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            }
          }
        },

How can I get the data in the index displayed as two different elements within a json array , instead of comma seperated values

In my ndjson the data is passed as below

"BrandName":[{"brand":"Optical"},{"brand":"Sun"}]

2

Hi

Any updates on this

3

Hello, having them as an array won't be a good solution since array functionality is not good in Kibana as you won't be able to aggregate on that field.
How I would recommend is to use the brand directly as "optical sun" and set it as text and not keyword. This way any search on "optical" or "sun" would find that item, since the field will be analyzed.

4

Thanks @Marius_Dragomir

But in my Import NDJSON file , how will I pass different values for Brand

This is one sample record as below. So for one single record I have multiple values for brand. How can i pass both "sun" and "optical" as text in the same record without an array

{"Fiscal Week":"FW01","StartTime":"Dec 31 2019 00:00:00","EndTime":"Jan 1 2020 01:00:00","month":"Jan","Year":"2020","Summary":"Store power failure","Description":"Natural phenomena such as wind, lightning, freezing rain","AppName":"xxx","BrandName":[{"brand":"Optical"},{"brand":"Sun"}]
,"Orgs":[{"Org":"0"},{"Org":"20"}],"Banners":[{"Banner":"zzz"},{"Banner":"yyy"}],"Regions":[{"Region":"USA"},{"Region":"CAN"}],"Total Stores":"10","Store Count":"2","Severity":"98","Outage Hours":25,"App Uptime in %":50,"Actual Store SLA %":100,"Actual Severity SLA %":98 ,"SLA Store Comp":100,"SLA Comp Severity":100}

As per what you said pass the brand as "brand":"Optical Sun" in the ND JSON file?

5

yes, that would work. The data visualizer isn't really the intended way to ingest a lot of data, it's there for a small number of records to see how things work.
Normally you'd go through elasticsearch pipelines or Logstash and you could merge/flatten that array that you have at ingest in the pipelines.
If you insist on using the data visualizer you have to do the merge yourself before importing the data in the Data Visualizer.

6

Thanks @Marius_Dragomir is there an example where I can see this merge if flatten the Json . I see there is a pipeline option in the advance setting in data visualiser . How can I flatten the Json for this intended purpose ? I tried to use option sun in the brand section but it goes as a single text with both the values and is considering as a single value not two different values

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.