# Timeouts when indexing

**URL:** <https://discuss.elastic.co/t/timeouts-when-indexing/124591>\
**Category:** Elasticsearch\
**Created:** [March 19, 2018, 3:13pm UTC](https://discuss.elastic.co/t/timeouts-when-indexing/124591 "2018-03-19T15:13:52Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![mgo](https://avatars.discourse-cdn.com/v4/letter/m/3e96dc/32.png) [@mgo](https://discuss.elastic.co/u/mgo)\
**Post date:** [March 19, 2018, 3:13pm UTC](https://discuss.elastic.co/t/timeouts-when-indexing/124591/1 "2018-03-19T15:13:52Z")

</div>

Dear All,

I am using ES for logging requests/responses to an external API. In case of a problem, these logs are searched to resolve the issue.  
99% of requests to ES are index/update queries.  
Most of the time it works, but sometimes (3-5 times per 24h) the ES server is not responsing within 3 seconds (current limit for timeout).  
The timeouts mean lost data, which is not acceptable. On the other hand increasing timeout to more than 3 seconds is also not acceptable. In case of network issues and unreachability of ES from the web server - the request to ES should timeout relatively fast and the website should be usable. I would even prefer to set it at 1s.

After studying information from the Monitoring - I discovered that the timeouts correlate with spikes in:

- Indexing Time - jump from less than 1s to over 6s  
10:06  
 ![indexing-time](https://us1.discourse-cdn.com/elastic/original/3X/c/1/c14bb2762bacf6ccd32e732f6cf71c9afc327987.png)
- Refresh Time - jump from around 1s to over 5s  
10:06  
 ![refresh-time](https://us1.discourse-cdn.com/elastic/original/3X/6/b/6b06c38cd5d39b9245014945d9fb7472e664a305.png)
- Request Time - jump from ~0 to 6000ms  
 ![request-time](https://us1.discourse-cdn.com/elastic/original/3X/e/c/ec06cbdc67fc8af255c3c635d58897e525d518d0.png)  
10:06

and a little bit in with:

- Indexing Latency  
10:06  
 ![indexing-latency](https://us1.discourse-cdn.com/elastic/original/3X/a/7/a70e456a78cabb374fd9abfb2ff0e06f7e1434ce.png)
- Latency  
10:06  
 ![latency](https://us1.discourse-cdn.com/elastic/original/3X/6/9/697f83d59fa18dc0794098ab824134c89c4e0723.png)

The times of spikes are random. The CPU or Memory usage does not spike.  
Full monitoring below:  
Overview 10:06

 ![overview](https://us1.discourse-cdn.com/elastic/original/3X/b/3/b3e68d7b3c8341ca40c38232608bac4d34fcaacd.png)  
Index Overview 10:06  
 ![index-overview](https://us1.discourse-cdn.com/elastic/original/3X/d/8/d80acb38b6b6e3d825e1d541ce96e01c4f03a51e.png)  
Index Advanced 10:06  
 ![index-advanced](https://us1.discourse-cdn.com/elastic/original/3X/7/4/74f1fc60264f68f8e67923471b63ee1738d73a9c.png)  
Node Overview 10:06  
 ![node-overview](https://us1.discourse-cdn.com/elastic/original/3X/f/b/fb599453fdbe316aed1c7453a3ee143cc9668506.png)  
Node Advanced 10:06  
 ![node-advanced](https://us1.discourse-cdn.com/elastic/original/3X/3/c/3cd82c6b0ff0d54262212a264d04dccfcc9d83cc.png)  
 ![node-advanced2](https://us1.discourse-cdn.com/elastic/original/3X/f/b/fb46bd3cc15d1e7795e0a206320e13ddb7e24f61.png)

So far I changed with no success:

indices.memory.index\_buffer\_size: 50%  
thread\_pool.index.queue\_size: 5000

On index creation:  
refresh\_interval: 30s

The index options:

```
{
	"state": "open",
	"settings": {
	    "index": {
	        "refresh_interval": "30s",
	        "number_of_shards": "1",
	        "provided_name": "requests_2018-03-19",
	        "max_result_window": "1000000",
	        "creation_date": "1521414001578",
	        "number_of_replicas": "0",
	        "uuid": "xd75vemBTdmZS6Us6RT0Dw",
	        "version": {
	            "created": "5060599"
	        }
	    }
	}
}

```

Machine is: 4 cores/8GB RAM/SSD

Do you have any ideas what else I could tune to overcome the timeout issue?  
All the indexed documents are similar in size. For every index operation (API request) there usually is also a following update operation (API response). I cannot understand the reason for the spikes. I also wonder why Index Queue is always 0.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 16, 2018, 3:27pm UTC](https://discuss.elastic.co/t/timeouts-when-indexing/124591/3 "2018-04-16T15:27:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.