Error sending data to elasticsearch

I have come accorss with the following error. Is there anyone who have ideas with that?

2016/06/03 10:49:02.295137 client.go:190: INFO Bulk item insert failed (i=49, status=503): {"type":"unavailable_shards_exception","reason":"[topbeat-2016.06.03][2] primary shard is not active Timeout: [1m], request: [BulkShardRequest to [topbeat-2016.06.03] containing [10] requests]"}

A 503 can indicate an overload which means the thread pools are busy and so the request is rejected.

Yes I found it is the disk size problem
What files, like old data, and where should I delete in elasticsearch to make the beats feed data again?
And how can I make the data to just keep for some time and delete automatucally after a certain period as they become old data


You can use curator to schedule delete

There are some example

You might want to delete index older than 'x' days for instance

When I tried to run the command, it comes across with the following issues.

[Administrator@localhost elasticsearch]$ curator --host delete indices --older-than 2 --time-unit days --timestring '%Y.%m.%d'
Traceback (most recent call last):
File "/usr/bin/curator", line 5, in
from pkg_resources import load_entry_point
ImportError: No module named pkg_resources

I am now coming across with the problem like that

Exception in thread "main" java.nio.file.FileSystemException: /tmp/2427469249444301473.tmp: No space left on device
at sun.nio.fs.UnixException.translateToIOException(
at sun.nio.fs.UnixException.rethrowAsIOException(
at sun.nio.fs.UnixException.rethrowAsIOException(
at sun.nio.fs.UnixFileSystemProvider.newByteChannel(
at java.nio.file.Files.newByteChannel(
at java.nio.file.Files.createFile(
at java.nio.file.TempFileHelper.create(
at java.nio.file.TempFileHelper.createTempFile(
at java.nio.file.Files.createTempFile(
at org.elasticsearch.bootstrap.Security.selfTest(
at org.elasticsearch.bootstrap.Security.configure(
at org.elasticsearch.bootstrap.Bootstrap.setupSecurity(
at org.elasticsearch.bootstrap.Bootstrap.setup(
at org.elasticsearch.bootstrap.Bootstrap.init(
at org.elasticsearch.bootstrap.Elasticsearch.main(
Refer to the log for complete error details.