GET Document from Index with Kibana API

[Jonathan_Emami]

Hi,
I'm writing here cause I've read through all of the Kibana REST API, but I still have not found an answer to my question REST API | Kibana Guide [8.6] | Elastic

I basically want to access a document in the index, in the same way as the Analytics > Discover function, and I'd like to test the request through Postman. In order to make this kind of request f, from my understanding, we need the following pre-requisites:

• The right user privileges
  What user privileges' do we need in order to make this request?

• The right URL
  What URL do we use to access a document as an external user? I couldn't find it anywhere

• The document ID
  I found the ID to my index under Discover > Inspect

• A Request Body
  I'd like to query for different error types in my applications stack trace. Under Discover > Inspect I found the following request boy:

{
    "track_total_hits": false,
    "sort": [
      {
        "@timestamp": {
          "order": "desc",
          "unmapped_type": "boolean"
        }
      }
    ],
    "fields": [
      {
        "field": "@timestamp",
        "format": "strict_date_optional_time"
      },
      {
        "field": "error.exception.code"
      },
      {
        "field": "error.exception.handled"
      },
      {
        "field": "error.exception.message"
      },
      {
        "field": "error.exception.module"
      },
      {
        "field": "error.exception.type"
      }
    ],
    "size": 500,
    "version": true,
    "script_fields": {},
    "stored_fields": [
      "*"
    ],
    "runtime_mappings": {},
    "_source": false,
    "query": {
      "bool": {
        "must": [],
        "filter": [
          {
            "bool": {
              "should": [
                {
                  "match_phrase": {
                    "service.name": "adresseserviceprod"
                  }
                }
              ],
              "minimum_should_match": 1
            }
          },
          {
            "range": {
              "@timestamp": {
                "format": "strict_date_optional_time",
                "gte": "2023-02-04T23:00:00.000Z",
                "lte": "2023-02-07T09:31:19.662Z"
              }
            }
          },
          {
            "exists": {
              "field": "error.exception.type"
            }
          }
        ],
        "should": [],
        "must_not": []
      }
    },
    "highlight": {
      "pre_tags": [
        "@kibana-highlighted-field@"
      ],
      "post_tags": [
        "@/kibana-highlighted-field@"
      ],
      "fields": {
        "*": {}
      },
      "fragment_size": 2147483647
    }
  }

Does somebody know a solution to this? It feels like it shouldn't be this hard to make a basic call to the index...

[Ayush_Mathur]

Hello @Jonathan_Emami , welcome to the community !

@Jonathan_Emami

What user privileges' do we need in order to make this request?

You or the account used by your application will need read access to the indices which will be queried upon.

@Jonathan_Emami

What URL do we use to access a document as an external user?

You essentially need the Elasticsearch URL which is accessible for any client, may be kibana.yml can be referred where you need to specify elasticsearch.hosts with the actual ES URL.

@Jonathan_Emami

I found the ID to my index under Discover > Inspect

If you are querying over application, it would pretty hard to find the ID of each and every document first. Instead, try using DevConsole to prepare a DSL query that can be used to retrieve desired documents. For starters, refer: The search API | Elasticsearch Guide [8.11] | Elastic

@Jonathan_Emami

I'd like to query for different error types in my applications stack trace.

If the error types exist as a separate field in your documents, it's pretty simple to find out the unique values.
In query part, a bool -> must -> exists query will suffice where existence of message.error.type will be checked in your documents, with range on timestamp, i.e. period over which you want to search the documents on.
The resulting payload documents can then be grouped using terms aggregation on `message.error.type.keyword field which will create buckets / array of all unique values and doc_count for each of the value.

[Jonathan_Emami]

Thank you Ayush for a quick response!

I'll contact my Kibana administrator for the information below. I hope that'll be enough to resolve my problems

@Ayush_Mathur

You or the account used by your application will need read access to the indices which will be queried upon.

My account already has reading privileges to the 'space' where the data view is located in, so I guess that should be enough? Maybe I need the 'monitor' privilege, see 'Indices privileges'.

@Ayush_Mathur

You essentially need the Elasticsearch URL which is accessible for any client, may be kibana.yml can be referred where you need to specify elasticsearch.hosts with the actual ES URL.

I see. My best guess for the URL was (from this page)
<kibana host>:<port>/s/<space_id>/api/data_views/data_view/<id>
but it never returned a response for my request body. I'll try to find the correct URL in the kibana-yml file, thanks.

[Ayush_Mathur]

@Jonathan_Emami monitor privilege is not required in this case since you are not drilling down into segments and index statistics rather just want to retrieve the documents using search.

About the URL, in some cases where you need to embed Kibana UI in your application like Selenium test cases, you can use the kibana_host:kibana_port URIs. However, to search for documents from an index, you need to hit ES APIs and not Kibana's. Also, the URL that you've mentioned is about data views or indices themselves, it cannot be used for documents I believe.