First query is slow after every short period of time

Hi,

I am using Elasticsearch version 7.6.2 inside a docker container and i use logstash to index data to my ES Node but when i query the ES indexes after a short amount of time, it takes a significant time to return search results but subsequent queries are very fast.
For e.g. if time difference between two queries is 20 min, it takes up-to 3-4 secs to get back the results but if i query after 30 secs, it takes around 0.1 sec. What is more worrying that if i don't query it for 2-3 days, it took around 10secs to get back the results.
I have around 3300 docs in my 4 indexes and 3000 of them in a single index.
My query looks like this :

{
	"query": {
		"bool": {
			"filter": {
				"term": {
					"orgid.keyword": "43c5974b-c6e2-418f-b41c-567450ca7e9f"
				}
			},
			"must": {
				"query_string": {
					"analyze_wildcard": true,
					"fields": ["name"],
					"query": "*s_8*",
					"rewrite": "scoring_boolean"
				}
			}
		}
	}
}

and the index template :

{
    "template": "object",
    "index_patterns" : ["ap*", "ar*", "dg*", "de*"],
    "order" : 1,
    "settings" : {
        "number_of_shards" : 1,
        "search.idle.after" : "1800s",
        "refresh_interval" : "65s",
        "analysis": {
            "analyzer": {
                "case_insensitive": {
                    "tokenizer": "keyword",
                    "filter": [
                    "lowercase"
                    ]
                }
            }
        }
    },
    "mappings": {
        "properties": {
              "name": {
                "type" : "text",
                "analyzer": "case_insensitive",
                "fielddata": true
            }
           
        }
     }
}

i index docs after every 60sec. I tried search.idle.after, refresh_interval and fielddata but they didn't help me much