ES eventually falls in timeouts

(Andrey Afanasyev) #1

I have server with 32Gb RAM under Windows Server 2012 R2 Standard.
I have installed on server:

  1. ES 1.6.0 - takes 8Gb of RAM, configured as single node.
    ES configuration: xxxxx
bootstrap.mlockall: true xxxxx
transport.tcp.port: xxxxx
http.port: xxxxx 10s 1s
index.indexing.slowlog.threshold.index.warn: 10s
monitor.jvm.gc.young.warn: 1000ms
monitor.jvm.gc.old.warn: 10s
script.inline: on
script.indexed: on

  1. MS SQL Server 2008R2 - takes 16Gb of RAM.
    When I just started ES - all is fine, works without any timeouts.
    But after 3-4 days I'm starting get timeouts during different ES operations (search, msearch, bulk insert).
    During getting timeouts BigDesk shows that do not exist any problems with HeapMemory (~5-6 Gb out of 8Gb is used), CPU (cpu usage ~30-40%), Search requests per second ( 50-80 requests per second).
    I turned on slow logging and noticed that number of records in elasticsearch_index_search_slowlog.* files usually grows day by day and some queries are executed in 1-2 minutes.
    The volume of indexed documents is the same in 3-4 days approximately, so I do not think that it could be a reason.
    If I restart ES - it helps, but situation is repeated in 3-4 days.
    Any hints what could it be?
    On what should I pay attention in order to recognize possible problems?

(Mark Walkom) #2

How much data is in ES?

(Andrey Afanasyev) #3

About 2 millions of documents, it takes about 10Gb on hard disk.

(system) #4