# Logstash unable to map field to es **URL:** \ **Category:** Logstash\ **Created:** [August 8, 2018, 9:20pm UTC](https://discuss.elastic.co/t/logstash-unable-to-map-field-to-es/143599 "2018-08-08T21:20:22Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![pooch](https://avatars.discourse-cdn.com/v4/letter/p/dbc845/32.png) [@pooch](https://discuss.elastic.co/u/pooch)\ **Post date:** [August 8, 2018, 9:20pm UTC](https://discuss.elastic.co/t/logstash-unable-to-map-field-to-es/143599/1 "2018-08-08T21:20:22Z")
I am using a multiline config in filebeat to send logs as one line, not sure if that is the cause of this. I am currently getting this error when indexing certain log files. **response=\>{"index"=\>{"\_index"=\>"filebeat-2018.08.08", "\_type"=\>"log", "\_id"=\>"AWUbQVOf54TmqKKkRQHo", "status"=\>400, "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"object mapping for [source] tried to parse field [source] as object, but found a concrete value"}}}}** Here is the full output. ```auto [2018-08-08T20:36:30,765][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"filebeat-2018.08.08", :_type=>"log", :_routing=>nil}, 2018-08-08T20:36:23.367Z vdpsidxtst05 2018-08-08 20:36:22.512 ERROR (qtp401424608-643) [] o.a.s.s.HttpSolrCall null:org.apache.solr.common.SolrException: Core with name 'cpcr' already exists. at org.apache.solr.core.CoreContainer.create(CoreContainer.java:839) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:88) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:370) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:388) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:748) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:729) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:510) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:298) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:534) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:745) ], :response=>{"index"=>{"_index"=>"filebeat-2018.08.08", "_type"=>"log", "_id"=>"AWUbQVOf54TmqKKkRQHo", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"object mapping for [source] tried to parse field [source] as object, but found a concrete value"}}}} { ``` Here is the field mapping for **source** ```auto "source": { "properties": { "class": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } } ``` LS creates a new index everyday for that days data. I ran into this yesterday and deleted the index and then it processed the record but that was a temporary fix. I would much appreciate pointers on how to manage this. Thanks! ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [August 9, 2018, 7:48am UTC](https://discuss.elastic.co/t/logstash-unable-to-map-field-to-es/143599/2 "2018-08-09T07:48:29Z")
For some reason the `source` field is mapped as an object (with a `class` subfield) and then you can't index documents where `source` contains a concrete value. Which documents contain `source.class` fields? Are those incorrect? Or are the documents where `source` is a concrete value incorrect? Both can't be right. ---
**Author:** ![pooch](https://avatars.discourse-cdn.com/v4/letter/p/dbc845/32.png) [@pooch](https://discuss.elastic.co/u/pooch)\ **Post date:** [August 9, 2018, 6:17pm UTC](https://discuss.elastic.co/t/logstash-unable-to-map-field-to-es/143599/3 "2018-08-09T18:17:02Z")
Thank you for taking the time to reply magnusbaeck! I am very new to all things ELK so I am attempting to decipher your questions. I will update the thread once I have something of value. ---
**Author:** ![pooch](https://avatars.discourse-cdn.com/v4/letter/p/dbc845/32.png) [@pooch](https://discuss.elastic.co/u/pooch)\ **Post date:** [August 9, 2018, 7:22pm UTC](https://discuss.elastic.co/t/logstash-unable-to-map-field-to-es/143599/4 "2018-08-09T19:22:41Z")
Is this helpful? ```auto sbdps-devops07{93}$ curl --user elastic:changeme -X GET "localhost:9200/_search?pretty=yes" -H 'Content-Type: application/json' -d' { "query": { "exists" : { "field" : "source" }}}' { "took" : 920, "timed_out" : false, "_shards" : { "total" : 110, "successful" : 110, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : 22129, "max_score" : 1.0, "hits" : [ { "_index" : "filebeat-2018.08.02", "_type" : "log", "_id" : "AWT38XgvEhcjGUNcD1xa", "_score" : 1.0, "_source" : { "environment" : "tst", "logtype" : "application_log", "@timestamp" : "2018-08-02T00:00:50.887Z", "offset" : 1505290, "@version" : "1", "input_type" : "log", "beat" : { "name" : "vdpsidxtst05", "hostname" : "vdpsidxtst05", "version" : "5.5.0" }, "host" : "vdpsidxtst05", "source" : "/local/mnt/logs/filebeat/apps/solr/solr-qdisk-new.log", "message" : "\tcommit{dir=NRTCachingDirectory(MMapDirectory@/prj/solr_master01/vdpsidxtst05/solr-qdisk-new/data/qdisk/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7d74fad9; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4s77,generation=223171}", "type" : "log", "tags" : [ "beats_input_codec_plain_applied", "_grokparsefailure" ] } }, { "_index" : "filebeat-2018.08.02", "_type" : "log", "_id" : "AWT38XkXEhcjGUNcD1xg", "_score" : 1.0, "_source" : { "logtype" : "application_log", "environment" : "tst", "@timestamp" : "2018-08-02T00:00:50.887Z", "offset" : 1520205, "@version" : "1", "input_type" : "log", "beat" : { "name" : "vdpsidxtst05", "hostname" : "vdpsidxtst05", "version" : "5.5.0" }, "host" : "vdpsidxtst05", "source" : "/local/mnt/logs/filebeat/apps/solr/solr-qdisk-new.log", "message" : "\tcommit{dir=NRTCachingDirectory(MMapDirectory@/prj/solr_master01/vdpsidxtst05/solr-qdisk-new/data/qdisk/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7d74fad9; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4s7a,generation=223174}", "type" : "log", "tags" : [ "beats_input_codec_plain_applied", "_grokparsefailure" ] } }, ``` ---
**Author:** ![pooch](https://avatars.discourse-cdn.com/v4/letter/p/dbc845/32.png) [@pooch](https://discuss.elastic.co/u/pooch)\ **Post date:** [August 9, 2018, 7:47pm UTC](https://discuss.elastic.co/t/logstash-unable-to-map-field-to-es/143599/5 "2018-08-09T19:47:53Z")
This may be helpful as well?? ```auto { "_index": "filebeat-2018.08.09", "_type": "log", "_id": "AWUc16lD54TmqKKkR8pR", "_version": 1, "_score": null, "_source": { "offset": 1809078, "level": "ERROR", "profile": "tst", "logger": "com.blah.ems.search.solr.agile.AgileDocumentIndexer", "module": "api", "input_type": "log", "source": { "file": "AgileDocumentIndexer.java", "method": "index", "class": "com.blah.ems.search.solr.agile.AgileDocumentIndexer", "line": 68 }, ``` ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [August 9, 2018, 9:10pm UTC](https://discuss.elastic.co/t/logstash-unable-to-map-field-to-es/143599/6 "2018-08-09T21:10:19Z")
OK, so in one you have ``` "source" : "/local/mnt/logs/filebeat/apps/solr/solr-qdisk-new.log" ``` and in the other you have ``` "source": { "file": "AgileDocumentIndexer.java", "method": "index", "class": "com.blah.ems.search.solr.agile.AgileDocumentIndexer", "line": 68 } ``` You cannot have the source field be an object in some documents and text in other documents in the same index. You would have to use multiple indexes. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 6, 2018, 9:10pm UTC](https://discuss.elastic.co/t/logstash-unable-to-map-field-to-es/143599/7 "2018-09-06T21:10:20Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.