Failed to sync translog java.io.IOException: Input/output error


(Ziv) #1

Hey,

I'm receiving this error quite often ( every 5-10 seconds) from my indices and even marvel index.

From my indices when I bulk index 200-1000 documents from a single machine in a single thread.

And from marvel when the system is idle:
[2015-08-24 13:24:26,561][WARN ][index.gateway.local ] [secdo_test_1] [.marvel-2015.08.24][0] failed to sync translog
java.io.IOException: Input/output error
at sun.nio.ch.FileDispatcherImpl.force0(Native Method)
at sun.nio.ch.FileDispatcherImpl.force(FileDispatcherImpl.java:76)
at sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:386)
at org.elasticsearch.index.translog.fs.SimpleFsTranslogFile.sync(SimpleFsTranslogFile.java:162)
at org.elasticsearch.index.translog.fs.FsTranslog.sync(FsTranslog.java:440)
at org.elasticsearch.index.gateway.local.LocalIndexShardGateway$Sync$1.run(LocalIndexShardGateway.java:378)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

I'm running 1.7.1 and its an EC2 single node with EBS volumes raid0.

How bad is this?

Best regards,


(Mark Walkom) #2

Doesn't look good, can you run a FS check?


(Ziv) #3

thanks for the tip.

I used mdadm raid0 on amazon EBS.
Once I changed the database to use only one EBS volume instead of mdadm it started working properly again.


(system) #4