Can't find data when doing simple KQL query ( can see the log in general search)

im using the filebeat-* index .
when i watch the logs in discover i can see logs from the pods
when i try to create simple query to get logs by namespace/name/container name
i don't get any result
i follow this thread post :

i tried :

kubernetes.namespace_name.keyword : my_name_space and kubernetes.pod_name.keyword : prod* 

(which i can see that they exist )

but when i do simple query like :

agent.name: ip-11-Xxx-0-xxx.ec2.internal

it does return results

what do i missing here ?

What version of Kibana are you running? Can you share the mapping of your index?

In general, make sure your search bar is set to KQL, not Lucene (you can see this to the right of input)

Thank you for answering
After digging deeper into the problem, i found out that the ELK + Filebeat stoped to work from 1/1/2021
Looking at the logs in one of the filebeat pods i can see this:

2021-01-04T10:10:52.754Z        DEBUG   [add_cloud_metadata]    add_cloud_metadata/providers.go:129     add_cloud_metadata: fetchMetadata ran for 2.351101ms
2021-01-04T10:10:52.754Z        INFO    [add_cloud_metadata]    add_cloud_metadata/add_cloud_metadata.go:93     add_cloud_metadata: hosting provider type detected as openstack, metadata={"ava
ilability_zone":"us-east-1c","instance":{"id":"i-08f536567bd9945df","name":"ip-10-101-2-178.ec2.internal"},"machine":{"type":"m5.2xlarge"},"provider":"openstack"}
2021-01-04T10:10:52.755Z        DEBUG   [processors]    processors/processor.go:120     Generated new processors: add_cloud_metadata={"availability_zone":"us-east-1c","instance":{"id":"i-08f5
36567bd9945df","name":"ip-10-101-2-178.ec2.internal"},"machine":{"type":"m5.2xlarge"},"provider":"openstack"}, add_docker_metadata=[match_fields=[] match_pids=[process.pid, process.ppid]]    
2021-01-04T10:10:52.755Z        INFO    instance/beat.go:392    filebeat stopped.
2021-01-04T10:10:52.755Z        ERROR   instance/beat.go:956    Exiting: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (pat
h.data).
Exiting: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data).

as you can see the filebeat stopped with an error :

data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data).

After searching the problem in github/forum i found this :

Which looks like my problem,
Im using the default filebeat-kubernetes.yaml , and there is no information in your docs on how to add unique paths in the filebeat-kubernetes.yaml
where do i add them and how do i make them unique?
Thanks
@flash1293

As this is a beats question, could you post it in the beats category? https://discuss.elastic.co/c/elastic-stack/beats/28

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.