Example documents

graph

(reza sadoddin) #1

The graph has the nice feature of "show example documents". However, I just can see the ID's of documents. No information is shown for other fields of documents. Does that mean the documents should be returned by IDs in Kibana or ES by another query or I am missing something?

Thanks,


(Mark Harwood) #2

The "ID" header should be a drop-down letting you select from a choice of other fields.


(reza sadoddin) #3

I see the drop-downs and I tried to switch to other fields, but nothing is shown after selecting other fields. I am sure the fields have correct values based on the query I run through ES.


(reza sadoddin) #4

This is observed on Chrome running on Windows.


(Mark Harwood) #5

One issue is that the list of the fields in the drop down are those defined in the mapping, some of which are in the index but not present in the original _source json. These sorts of fields cannot be shown. To help debug this try open the dev tools in Chrome, switch to the "Network" tab then hit the graph UI's "Show example docs" button. In the Chrome dev tools panel you should see a network call to "getExampleDocs" and can see the raw JSON response including the hits. This should help reveal what content should be displayable. However anything but simple root-level fields (e.g. nested arrays of objects) might have some challenges.

Going forward I want to replace this primitive doc viewer with the ability to drill down using any pre-saved Kibana visualisation (tables, maps, timelines..)


#6

Following up - I would like to return all documents reflected in each edge - is there a REST query for this?


(Mark Harwood) #7

Yes. First a tip to get hold of the query JSON - when you have some vertices selected in the workspace perform the following steps:

Having got the query pass that as the body in a search request as described here: https://www.elastic.co/guide/en/elasticsearch/guide/2.x/query-dsl-intro.html


(Mark Harwood) #8

Actually, one extra tip...
Having copied the query body using the approach in my last comment you can paste the query JSON into other Kibana text inputs to filter dashboards or visualizations as shown below:


#9

Ok - thanks. And is this functionally equivalent to using the getExampleDocs endpoint as described above, using a top_hits size of infinity?


(Mark Harwood) #10

The pie chart is (here it is showing aggregated doc counts for 2 terms "male" and "female").

To extract very large amounts of individual docs you need to look at this API: https://www.elastic.co/guide/en/elasticsearch/guide/2.x/scroll.html


#11

Hmm - when I query api/graph/getExampleDocs with this:

json: {
      index:"jdbc",
      body: {
          query:{ 
              bool: {
                  should: [{
                      term: {
                              body: "[ORIGINAL GRAPH QUERY]"
                          }
                      }
                  ]
              }
          },
          size: 0,
          aggs: {
              sample: {
                  sampler: {},
                  aggs: {
                  topHits: {
                      top_hits: {
                              size:10000
                          }
                      }
                  }
              }
          }
      }
    }

(which is the same request that goes out when I check the network tab, with the addition of the higher top_hits value) I get the actual documents. What are these then?


(Mark Harwood) #12

A way of getting 1000 docs but not an advisable way of getting 1,000,0000,000 docs.


(system) #13