Search docs not return 0 hits

Elastic Stack Elasticsearch
1

Hi, I´m trying to search some docs, in Kibana using discover i have 2 hits
with the following timestamp: @timestamp
Apr 3, 2024 @ 03:27:23.000 and Apr 3, 2024 @ 03:26:56.000 for a doc with a field "sender": "TEED-NEC455"

image
image1905×570 85.1 KB

but, when I try using the search API as follows doesn´t bring anything relevant

{
  "query": {
    "bool": {
      "must": [
        {
          "range": {
            "date": {
              "gte": "2024-04-03T03:25:00Z",
              "lte":  "2024-04-03T03:30:00Z"
            }
          }
        },
        {
          "term": {
            "sender": "TEED-NEC455"
          }
        }
      ]
    }
  }
}

result I get from the API search:

{
    "took": 0,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 0,
            "relation": "eq"
        },
        "max_score": null,
        "hits": []
    }
}

Thanks for your help!

2

Welcome!

What's the mapping for this index?

1 Like
3

This is the mapping:

{
  "mappings": {
    "properties": {
      "@timestamp": {
        "type": "date"
      },
      "device": {
        "properties": {
          "description": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      },
      "geo": {
        "properties": {
          "location": {
            "type": "geo_point"
          }
        }
      },
      "host": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "identifier": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "index": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "info": {
        "properties": {
          "areas": {
            "properties": {
              "circles": {
                "properties": {
                  "center": {
                    "properties": {
                      "latitude": {
                        "type": "float"
                      },
                      "longitude": {
                        "type": "float"
                      }
                    }
                  },
                  "radius": {
                    "type": "float"
                  }
                }
              },
              "description": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              }
            }
          },
          "category": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "certainty": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "direction": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "effective": {
            "type": "date"
          },
          "event": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "expires": {
            "type": "date"
          },
          "infodescription": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "language": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "objectid": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "objecttype": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "parameters": {
            "type": "nested",
            "properties": {
              "BRAND": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "CAMERAID": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "CAMERANAME": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "COLOUR": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "COUNTRY": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "CSAI_APP_VALIDATIONS": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "CUSTOMINFO": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "DEVICEID": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "DEVICEPROVIDER": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "DEVICETYPE": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "DIRECTION": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "EFFECTIVE": {
                "type": "date"
              },
              "EQUIPMENTTYPE": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "EXTERNALEVENTID": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "IMAGEHEIGHT": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "IMAGELEFT": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "IMAGETOP": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "IMAGEWIDTH": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "LATITUDE": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "LONGITUDE": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "OBJECTENTITY": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "OBJECTID": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "OBJECTTYPE": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "ORIGINTYPE": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "SCORE": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "SPEED": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "TIMESTAMP": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "VEHICLETYPE": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              }
            }
          },
          "parameters,LATITUDE": {
            "type": "float"
          },
          "parameters,LONGITUDE": {
            "type": "float"
          },
          "sent": {
            "type": "date"
          },
          "severity": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "urgency": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "vehicletype": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      },
      "messagetype": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "scope": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "sender": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "sent": {
        "type": "date"
      },
      "source": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "sourcetype": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "status": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "tenantId": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "time": {
        "type": "long"
      }
    }
  }
}

Thanks

4

The search request specifies the time range in UTC time and all timestamps in Elasticsearch are indexed in UTC timezone. Kibana does by default however translate timestamps into the local timezone so your query range may, depending on the timezone you are located in, posibly not match what you have specified in Kibana.

Which timezone are you in?

5

I Christian,
my timezone is GMT-3.
I tried to set the filter with +3 like this:

 "range": {
            "date": {
              "gte": "2024-04-03T06:25:00Z",
              "lte":  "2024-04-03T06:30:00Z"
            }
          }

but It didn´t work, so I´m doing something wrong :pensive:

{
    "took": 0,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 0,
            "relation": "eq"
        },
        "max_score": null,
        "hits": []
    }
}
6

Any chance daylight saving time might play in?

7

Change that part of the query:

    {
      "term": {
        "sender": "TEED-NEC455"
      }
    }

To

    {
      "term": {
        "sender.keyword": "TEED-NEC455"
      }
    }

You might also have to fix the date as @Christian_Dahlqvist said

8

Are you sure you are using the correct date field?

The mapping you shared does not have a field named date, so you do not have this field in your index.

You need to use an existing date field like @timestamp or info.effective for example, those fields are date fields that exists in your index, but you do not have any field named date.

Try this:

{
  "query": {
    "bool": {
      "must": [
        {
          "range": {
            "info.effective": {
              "gte": "2024-04-03T03:25:00Z",
              "lte":  "2024-04-03T03:30:00Z"
            }
          }
        },
        {
          "term": {
            "sender": "TEED-NEC455"
          }
        }
      ]
    }
  }
}
2 Likes
9

Kibana filter based on the @timestamp field by default, so unless you have overridden this you should probably change your range clause as follows:

"range": {
            "@timestamp": {
              "gte": "2024-04-03T06:25:00Z",
              "lte":  "2024-04-03T06:30:00Z"
            }
          }
1 Like
10

yep! I was setting wrong the time field,
so I changed it to this way:

{
  //"size": 42,
  "query": {
    "bool": {
      "must": [
        {
          "range": {
            "info.effective": {
              "gte": "2024-04-10T21:55:00-03:00",
              "lte": "2024-04-10T21:59:00-03:00"
            }
          }
        },
        {
          "match": {
            "sender": "TEED-NEC455"
          }
        }
      ]
    }
  }
}

and works like a charm! :smiley: