Elasticsearch python query does not display results

Elastic Stack Elasticsearch
1

Hello

I am trying to query certain content using python query.

Below is my code :

from flask import Flask, render_template, request
from elasticsearch import Elasticsearch
import os
es = Elasticsearch( ['http://*******:9200'], http_auth=('elastic', '*****') )
search_term = "ikea"

res = es.search(index='dba_docs_folder', body={"query" : {"match": {"content": search_term}}, "highlight" : {"pre_tags" : ["<b>"] , "post_tags" : ["</b>"], "fields" : {"content":{}}}}, size=6000)

print(res)

I get the below output with the above query :

{'took': 0, 'timed_out': False, '_shards': {'total': 1, 'successful': 1, 'skipped': 0, 'failed': 0}, 'hits': {'total': {'value': 0, 'relation': 'eq'}, 'max_score': None, 'hits': []}}

I intend to get results matching search_term. When I try to search the same using the kibana discover page, i get the desired results. But with the above query I get 0 hits.

Any suggestions on what am I doing wrong here ???

2

Hi @vikramaddagulla

Everything look fine, the only problem can be inside the query, double check that the query is really the same as the one you type in kibana.

As elastic don't return error, only an empty result it mean that you may hit another server that don't have the documents you search for (need to double check your configuration).

Can't really help more.

Not related with your problem but Just one question, do you really want 6000 results? if so, you can use scan helper it will help a lot or you can handle pagination.

https://elasticsearch-py.readthedocs.io/en/master/helpers.html?highlight=scan#elasticsearch.helpers.scan

3

Running on Kibana meaning : I just tried searching the term on the discover page :

image
image1672×411 42.5 KB

As you can see in the above screen there are about 7 hits. I expect the python query also to return the same result set.

Is my understanding correct ??

Regarding the 6000 result set, I had not provided any result set initially and then read few blogs which spoke about providing result set to display the correct data set.

4

Problem with discovery page is it's search in all fields so you need to go to dev tools and try this:

GET dba_docs_folder/_search
{
    "query": {"match": {"content": "ikea"}}
}

if you don't have result please provide the mapping of your fields and one document example.

GET dba_docs_folder/_mapping

5

Thanks for that. I tried searching the below on devtools

GET dba_docs_folder/_search
{
    "query": {"match": {"content": "ikea"}}
}

....and below is the response

{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 0,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  }
}
6

Below is the mapping in the JSON document for one of the hit via discover page.

There were some sensitive content and hence I had removed it and put *** in place of it.

{
  "_index": "dba_docs",
  "_type": "_doc",
  "_id": "e92453964e9c5269797bd2404db15",
  "_version": 1,
  "_score": null,
  "_source": {
    "content": ******************,
    "meta": {
      "date": "2020-05-19T17:08:11.000+00:00",
      "modifier": "*******",
      "created": "2006-09-16T04:00:00.000+00:00"
    },
    "file": {
      "extension": "xlsx",
      "content_type": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
      "created": "2020-06-06T16:05:37.000+00:00",
      "last_modified": "2020-06-06T16:05:37.000+00:00",
      "last_accessed": "2020-06-06T16:05:37.000+00:00",
      "indexing_date": "2020-06-06T16:07:09.980+00:00",
      "filesize": 21832,
      "filename": "ContactDetails.xlsx",
      "url": "file:///*********ContactDetails.xlsx"
    },
    "path": {
      "root": "2ef0e2bd54555269df2c7fc26f2335",
      "virtual": "/ContactDetails.xlsx",
      "real": "*****/ContactDetails.xlsx"
    }
  },
  "fields": {
    "file.last_modified": [
      "2020-06-06T16:05:37.000Z"
    ],
    "file.created": [
      "2020-06-06T16:05:37.000Z"
    ],
    "file.indexing_date": [
      "2020-06-06T16:07:09.980Z"
    ],
    "meta.created": [
      "2006-09-16T04:00:00.000Z"
    ],
    "file.last_accessed": [
      "2020-06-06T16:05:37.000Z"
    ],
    "meta.date": [
      "2020-05-19T17:08:11.000Z"
    ],
    "file.filename": [
      "ContactDetails.xlsx"
    ]
  },
  "highlight": {
    "content": [
      "***********************"
    ]
  },
  "sort": [
    1591459629980
  ]
}
7

Aaah...My bad...

I think I was searching in the wrong index..

I had to search under dba_docs
Instead...
I was searching under dba_docs_folder

1 Like
8

Just for information the mapping is not the result you provided, what you provided is a document.

something that you can get with this query:

GET dba_docs/_doc/e92453964e9c5269797bd2404db15

the mapping is the result of this query:

GET dba_docs_folder/_mapping

It give information on how your data are stored.
https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html

Good that your problem is solved! :grinning:

9

Thank you....Noted....I am still new to elastic and playing around...

The problem is now fixed and I am getting the required output.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.