# Search Request are Getting Socket Timeout **URL:** \ **Category:** Elasticsearch\ **Created:** [November 11, 2020, 7:43am UTC](https://discuss.elastic.co/t/search-request-are-getting-socket-timeout/255014 "2020-11-11T07:43:20Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![dillibabu.mekala](https://avatars.discourse-cdn.com/v4/letter/d/aca169/32.png) [@dillibabu.mekala](https://discuss.elastic.co/u/dillibabu.mekala)\ **Post date:** [November 11, 2020, 7:43am UTC](https://discuss.elastic.co/t/search-request-are-getting-socket-timeout/255014/1 "2020-11-11T07:43:20Z")
Hi Team, We have an index containing 365 days data. We have started reprocessing existing 1-Day data (i.e reprocessing includes-\> Insert New Data + then delete the Old data), we observed that while reprocessing is going on Search Queries are getting Socket Timeout because as reprocessing is going on Search Queries are taking more time than usual. Ex: Existing Data counts = 3370109 (for 1-Day) New Data = 3370109 (New Data for 1-Day) Reprocessing includes -\> Insert New Data (Its Spark Write) + then Delete Old Data (Its HighLevelRestClient Delete By Query) So while reprocessing is going on Search Queries are getting timeout and we should not increase the SocketTimeOut because its an user interface where should not see any latency in seeing the data. I am aware that Elastic Search allows for Data Reads while data process is going on, Could you please help us how to Read the data from elastic search cluster without Socket Timeout or without any latency for Search Requests (data read) while reprocessing is going on. Thank in advance. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 11, 2020, 8:01am UTC](https://discuss.elastic.co/t/search-request-are-getting-socket-timeout/255014/2 "2020-11-11T08:01:42Z")
Have you looked at identifying which resource is causing the timeouts? I would recommend first looking to see if disk I/O might be the limiting factor as it often is. Also look at CPU and for evidence of long or frequent GC in the logs. ---
**Author:** ![dillibabu.mekala](https://avatars.discourse-cdn.com/v4/letter/d/aca169/32.png) [@dillibabu.mekala](https://discuss.elastic.co/u/dillibabu.mekala)\ **Post date:** [November 11, 2020, 8:34am UTC](https://discuss.elastic.co/t/search-request-are-getting-socket-timeout/255014/3 "2020-11-11T08:34:57Z")
Here the problem is We are reading the Same reprocessing data while reprocess is going on. i.e data reads are taking time while our data reprocessing is going on which leads to the SocketTimeOut. Ex: Existing Data counts = 3370109 (for 1-Day) New Data = 3370109 (New Data for 1-Day) Can we increase I/O factor? if yes could you please help me how increase it? And is it going to help us in reading the data faster while reprocessing is going on? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 11, 2020, 8:37am UTC](https://discuss.elastic.co/t/search-request-are-getting-socket-timeout/255014/4 "2020-11-11T08:37:37Z")
First step is to identify what is limiting performance and causing latency. What type of storage do you have, e.g. local SSD? Have you looked at disk I/O, e.g. using `iostat`, while you are performing this operation? ---
**Author:** ![dillibabu.mekala](https://avatars.discourse-cdn.com/v4/letter/d/aca169/32.png) [@dillibabu.mekala](https://discuss.elastic.co/u/dillibabu.mekala)\ **Post date:** [November 11, 2020, 8:53am UTC](https://discuss.elastic.co/t/search-request-are-getting-socket-timeout/255014/5 "2020-11-11T08:53:47Z")
Hey Christian, We are using AWS Instance Types: c5.4xlarge.elasticsearch Data nodes storage type: EBS EBS volume type: Provisioned IOPS (SSD) EBS volume size: 500 GiB Provisioned IOPS: 5000 And "index.store.type": "niofs" ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 11, 2020, 9:11am UTC](https://discuss.elastic.co/t/search-request-are-getting-socket-timeout/255014/6 "2020-11-11T09:11:06Z")
> [@dillibabu.mekala](#): > > "index.store.type": "niofs" Do you have graphs showing disk I/O over time that you can correlate with the times the jobs run? Why have you got the `niofs` setting in place? ---
**Author:** ![dillibabu.mekala](https://avatars.discourse-cdn.com/v4/letter/d/aca169/32.png) [@dillibabu.mekala](https://discuss.elastic.co/u/dillibabu.mekala)\ **Post date:** [November 11, 2020, 10:02am UTC](https://discuss.elastic.co/t/search-request-are-getting-socket-timeout/255014/7 "2020-11-11T10:02:57Z")
Hey Christian, I don't have graphs, if you need i can run one more time & i can send you. And i have gone through the ES Docx and configured the 'niofs' > **[Store | Elasticsearch Reference \[7.9\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-store.html)** The NIO FS type stores the shard index on the file system (maps to Lucene `NIOFSDirectory` ) using NIO. It allows multiple threads to read from the same file concurrently. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 9, 2020, 10:02am UTC](https://discuss.elastic.co/t/search-request-are-getting-socket-timeout/255014/8 "2020-12-09T10:02:59Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.