特定のデータをElasticsearchに投入できない

s-hatanaka · February 1, 2017, 1:19am

MSOfficeで記載された数千ファイルをIngestAttachmentPluginを利用して、Elasticsearchへ投入し、検索するという用途で利用しています。
ただ下記の添付ファイルに限ってはElasticsearchのプロセスが強制終了し、Elasticsearchのログは何も出力されないという現象になってしまいます。バージョンは下記のとおりです。該当ファイルを用いると、必ずこの現象が発生します。ご確認いただけますでしょうか。

□添付ファイル

□バージョン情報
Elasticsearchのバージョン：5.1.1
IngestAttachmentPluginのバージョン：5.1.1

tsgkdt · February 1, 2017, 2:25am

確かに、指定のファイルを投入するとelasticsearchが落ちるようです。
出ていたログを見ますと、以下のようになっていました。NoClassDefFoundError、と。

[2017-02-01T02:15:50,737][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [] fatal error in thread [elasticsearch[Toi0qCP][index][T#1]], exiting
java.lang.NoClassDefFoundError: com/graphbuilder/curve/Point
さ        at java.lang.Class.getDeclaredConstructors0(Native Method) ~[?:1.8.0_92-internal]
        at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671) ~[?:1.8.0_92-internal]

このことから、以下で報告されている既存の不具合ではないかと思います。

github.com/elastic/elasticsearch

Fatal error with ingest-attachment plugin

opened 02:58PM - 09 Dec 16 UTC

closed 03:00PM - 16 Feb 17 UTC

chrduf

>bug :Data Management/Ingest Node

I am getting a fatal error when trying to index the attached stripped down docum…ent using the ingest-attachment plugin. This causes my cluster to reboot and does not give me an email notification. It looks to be having a problem with the embedded Visio diagram. Version: 5.1.1 (Elastic Cloud) ClusterId: 7e7501 Node: instance-0000000005 Plugin: ingest-attachment Link to problematic doc: https://1drv.ms/w/s!ApTXXtrEV_GGiosenEfoUSk1rRnuYA Error Information ``` Dec 8 21:42:34 ERROR org.elasticsearch.bootstrap.ElasticsearchUncaughtExceptionHandler i5@z0 ``` ``` [2016-12-08T21:42:34,628][ERROR][org.elasticsearch.bootstrap.ElasticsearchUncaughtExceptionHandler] fatal error in thread [elasticsearch[index][T#1]], exiting java.lang.NoClassDefFoundError: com/graphbuilder/curve/Point at java.lang.Class.getDeclaredConstructors0(Native Method) ~[?:1.8.0_72] at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671) ~[?:1.8.0_72] at java.lang.Class.getConstructor0(Class.java:3075) ~[?:1.8.0_72] at java.lang.Class.getDeclaredConstructor(Class.java:2178) ~[?:1.8.0_72] at org.apache.poi.xdgf.util.ObjectFactory.put(ObjectFactory.java:34) ~[?:?] at org.apache.poi.xdgf.usermodel.section.geometry.GeometryRowFactory.(GeometryRowFactory.java:39) ~[?:?] at org.apache.poi.xdgf.usermodel.section.GeometrySection.(GeometrySection.java:55) ~[?:?] at org.apache.poi.xdgf.usermodel.XDGFSheet.(XDGFSheet.java:77) ~[?:?] at org.apache.poi.xdgf.usermodel.XDGFShape.(XDGFShape.java:113) ~[?:?] at org.apache.poi.xdgf.usermodel.XDGFShape.(XDGFShape.java:107) ~[?:?] at org.apache.poi.xdgf.usermodel.XDGFBaseContents.onDocumentRead(XDGFBaseContents.java:82) ~[?:?] at org.apache.poi.xdgf.usermodel.XDGFMasterContents.onDocumentRead(XDGFMasterContents.java:66) ~[?:?] at org.apache.poi.xdgf.usermodel.XDGFMasters.onDocumentRead(XDGFMasters.java:101) ~[?:?] at org.apache.poi.xdgf.usermodel.XmlVisioDocument.onDocumentRead(XmlVisioDocument.java:106) ~[?:?] at org.apache.poi.POIXMLDocument.load(POIXMLDocument.java:190) ~[?:?] at org.apache.poi.xdgf.usermodel.XmlVisioDocument.(XmlVisioDocument.java:79) ~[?:?] at org.apache.poi.xdgf.extractor.XDGFVisioExtractor.(XDGFVisioExtractor.java:41) ~[?:?] at org.apache.poi.extractor.ExtractorFactory.createExtractor(ExtractorFactory.java:207) ~[?:?] at org.apache.tika.parser.microsoft.ooxml.OOXMLExtractorFactory.parse(OOXMLExtractorFactory.java:86) ~[?:?] at org.apache.tika.parser.microsoft.ooxml.OOXMLParser.parse(OOXMLParser.java:87) ~[?:?] at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280) ~[?:?] at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:120) ~[?:?] at org.apache.tika.parser.DelegatingParser.parse(DelegatingParser.java:72) ~[?:?] at org.apache.tika.extractor.ParsingEmbeddedDocumentExtractor.parseEmbedded(ParsingEmbeddedDocumentExtractor.java:102) ~[?:?] at org.apache.tika.parser.microsoft.ooxml.AbstractOOXMLExtractor.handleEmbeddedFile(AbstractOOXMLExtractor.java:311) ~[?:?] at org.apache.tika.parser.microsoft.ooxml.AbstractOOXMLExtractor.handleEmbeddedParts(AbstractOOXMLExtractor.java:202) ~[?:?] at org.apache.tika.parser.microsoft.ooxml.AbstractOOXMLExtractor.getXHTML(AbstractOOXMLExtractor.java:115) ~[?:?] at org.apache.tika.parser.microsoft.ooxml.OOXMLExtractorFactory.parse(OOXMLExtractorFactory.java:112) ~[?:?] at org.apache.tika.parser.microsoft.ooxml.OOXMLParser.parse(OOXMLParser.java:87) ~[?:?] at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280) ~[?:?] at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:120) ~[?:?] at org.apache.tika.Tika.parseToString(Tika.java:568) ~[?:?] at org.elasticsearch.ingest.attachment.TikaImpl$1.run(TikaImpl.java:94) ~[?:?] at org.elasticsearch.ingest.attachment.TikaImpl$1.run(TikaImpl.java:91) ~[?:?] at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_72] at org.elasticsearch.ingest.attachment.TikaImpl.parse(TikaImpl.java:91) ~[?:?] at org.elasticsearch.ingest.attachment.AttachmentProcessor.execute(AttachmentProcessor.java:72) ~[?:?] at org.elasticsearch.ingest.CompoundProcessor.execute(CompoundProcessor.java:100) ~[elasticsearch-5.1.1.jar:5.1.1] at org.elasticsearch.ingest.Pipeline.execute(Pipeline.java:58) ~[elasticsearch-5.1.1.jar:5.1.1] at org.elasticsearch.ingest.PipelineExecutionService.innerExecute(PipelineExecutionService.java:166) ~[elasticsearch-5.1.1.jar:5.1.1] at org.elasticsearch.ingest.PipelineExecutionService.access$000(PipelineExecutionService.java:41) ~[elasticsearch-5.1.1.jar:5.1.1] at org.elasticsearch.ingest.PipelineExecutionService$1.doRun(PipelineExecutionService.java:65) ~[elasticsearch-5.1.1.jar:5.1.1] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:527) ~[elasticsearch-5.1.1.jar:5.1.1] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-5.1.1.jar:5.1.1] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_72] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_72] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_72] Caused by: java.lang.ClassNotFoundException: com.graphbuilder.curve.Point at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[?:1.8.0_72] at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_72] at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:814) ~[?:1.8.0_72] at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_72] ... 47 more ```

discussにも同様の報告がありました。（上のissue元のようです）

s-hatanaka · February 7, 2017, 10:06am

情報共有ありがとうございます。

該当ファイルはサポートされなくなるというようなコメントもありますね。
ingest-attachment pluginには他の制約はあるのでしょうか？（どこかのページでサポート、非サポートファイル等の情報は載っていますでしょうか？）

system · March 7, 2017, 10:06am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.