Search docs not return 0 hits

JPXotox · April 12, 2024, 3:51pm

Hi, I´m trying to search some docs, in Kibana using discover i have 2 hits
with the following timestamp: @timestamp
Apr 3, 2024 @ 03:27:23.000 and Apr 3, 2024 @ 03:26:56.000 for a doc with a field "sender": "TEED-NEC455"

but, when I try using the search API as follows doesn´t bring anything relevant

{
  "query": {
    "bool": {
      "must": [
        {
          "range": {
            "date": {
              "gte": "2024-04-03T03:25:00Z",
              "lte":  "2024-04-03T03:30:00Z"
            }
          }
        },
        {
          "term": {
            "sender": "TEED-NEC455"
          }
        }
      ]
    }
  }
}

result I get from the API search:

{
    "took": 0,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 0,
            "relation": "eq"
        },
        "max_score": null,
        "hits": []
    }
}

Thanks for your help!

dadoonet · April 12, 2024, 5:23pm

Welcome!

What's the mapping for this index?

JPXotox · April 12, 2024, 6:02pm

This is the mapping:

{
  "mappings": {
    "properties": {
      "@timestamp": {
        "type": "date"
      },
      "device": {
        "properties": {
          "description": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      },
      "geo": {
        "properties": {
          "location": {
            "type": "geo_point"
          }
        }
      },
      "host": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "identifier": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "index": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "info": {
        "properties": {
          "areas": {
            "properties": {
              "circles": {
                "properties": {
                  "center": {
                    "properties": {
                      "latitude": {
                        "type": "float"
                      },
                      "longitude": {
                        "type": "float"
                      }
                    }
                  },
                  "radius": {
                    "type": "float"
                  }
                }
              },
              "description": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              }
            }
          },
          "category": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "certainty": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "direction": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "effective": {
            "type": "date"
          },
          "event": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "expires": {
            "type": "date"
          },
          "infodescription": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "language": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "objectid": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "objecttype": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "parameters": {
            "type": "nested",
            "properties": {
              "BRAND": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "CAMERAID": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "CAMERANAME": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "COLOUR": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "COUNTRY": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "CSAI_APP_VALIDATIONS": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "CUSTOMINFO": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "DEVICEID": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "DEVICEPROVIDER": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "DEVICETYPE": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "DIRECTION": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "EFFECTIVE": {
                "type": "date"
              },
              "EQUIPMENTTYPE": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "EXTERNALEVENTID": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "IMAGEHEIGHT": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "IMAGELEFT": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "IMAGETOP": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "IMAGEWIDTH": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "LATITUDE": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "LONGITUDE": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "OBJECTENTITY": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "OBJECTID": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "OBJECTTYPE": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "ORIGINTYPE": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "SCORE": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "SPEED": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "TIMESTAMP": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "VEHICLETYPE": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              }
            }
          },
          "parameters,LATITUDE": {
            "type": "float"
          },
          "parameters,LONGITUDE": {
            "type": "float"
          },
          "sent": {
            "type": "date"
          },
          "severity": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "urgency": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "vehicletype": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      },
      "messagetype": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "scope": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "sender": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "sent": {
        "type": "date"
      },
      "source": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "sourcetype": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "status": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "tenantId": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "time": {
        "type": "long"
      }
    }
  }
}

Thanks

Christian_Dahlqvist · April 12, 2024, 6:13pm

The search request specifies the time range in UTC time and all timestamps in Elasticsearch are indexed in UTC timezone. Kibana does by default however translate timestamps into the local timezone so your query range may, depending on the timezone you are located in, posibly not match what you have specified in Kibana.

Which timezone are you in?

JPXotox · April 12, 2024, 6:35pm

I Christian,
my timezone is GMT-3.
I tried to set the filter with +3 like this:

 "range": {
            "date": {
              "gte": "2024-04-03T06:25:00Z",
              "lte":  "2024-04-03T06:30:00Z"
            }
          }

but It didn´t work, so I´m doing something wrong

{
    "took": 0,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 0,
            "relation": "eq"
        },
        "max_score": null,
        "hits": []
    }
}

Christian_Dahlqvist · April 12, 2024, 7:09pm

Any chance daylight saving time might play in?

dadoonet · April 12, 2024, 11:19pm

Change that part of the query:

    {
      "term": {
        "sender": "TEED-NEC455"
      }
    }

To

    {
      "term": {
        "sender.keyword": "TEED-NEC455"
      }
    }

You might also have to fix the date as @Christian_Dahlqvist said

leandrojmp · April 13, 2024, 12:15am

Are you sure you are using the correct date field?

The mapping you shared does not have a field named date, so you do not have this field in your index.

You need to use an existing date field like @timestamp or info.effective for example, those fields are date fields that exists in your index, but you do not have any field named date.

Try this:

{
  "query": {
    "bool": {
      "must": [
        {
          "range": {
            "info.effective": {
              "gte": "2024-04-03T03:25:00Z",
              "lte":  "2024-04-03T03:30:00Z"
            }
          }
        },
        {
          "term": {
            "sender": "TEED-NEC455"
          }
        }
      ]
    }
  }
}

Christian_Dahlqvist · April 13, 2024, 6:41am

Kibana filter based on the @timestamp field by default, so unless you have overridden this you should probably change your range clause as follows:

"range": {
            "@timestamp": {
              "gte": "2024-04-03T06:25:00Z",
              "lte":  "2024-04-03T06:30:00Z"
            }
          }

JPXotox · April 15, 2024, 1:23pm

yep! I was setting wrong the time field,
so I changed it to this way:

{
  //"size": 42,
  "query": {
    "bool": {
      "must": [
        {
          "range": {
            "info.effective": {
              "gte": "2024-04-10T21:55:00-03:00",
              "lte": "2024-04-10T21:59:00-03:00"
            }
          }
        },
        {
          "match": {
            "sender": "TEED-NEC455"
          }
        }
      ]
    }
  }
}

and works like a charm!

system · May 13, 2024, 1:23pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Kibana Discover - No results found :\| Kibana	15	13153	July 6, 2017
Date range query returns no hits Kibana	6	731	January 26, 2022
Can't see documents in Kibana Elasticsearch	2	1105	May 6, 2017
Documents are stored on elasticsearch, can be checked via query, but Kibana does not find it in the discovery tab Kibana	5	849	June 30, 2020
Search API returns zero hits Elasticsearch	4	2859	July 15, 2019

Search docs not return 0 hits

Related topics