Need help for error " java.lang.IllegalArgumentException: Rejecting mapping update to [logstash-git] as the final mapping would have more than 1 type: [_doc, doc]"

[cheriemilk]

Hi team - need your help for this painful issue.

1. My ES Index template with index pattern is "logstash-git"

   {
   "template": "git_template",
   "index_patterns":["logstash-git"],
   "mappings": {
   "doc": {
   "properties": {
   "GIT_ORG": { "type": "text" },
   "GIT_REPOS": { "type": "text" },
   "COMMIT_SHA1": { "type": "text" },
   "COMMIT_AUTHOR": { "type": "text" },
   "COMMIT_DATE": { "type": "date",
   "format": "yyyy-MM-dd"},
   "COMMIT_COMMENT": { "type": "text" },
   "JIRA_KEY": { "type": "text" },
   "JIRA_ISSUE_TYPE": { "type": "text" },
   "JIRA_ISSUE_CATEGORY": { "type": "text" },
   "JIRA_PRIORITY": { "type": "text" },
   "JIRA_COMPONENTS": { "type": "text" },
   "JIRA_DEFECT_TYPE": { "type": "text" },
   "JIRA_TESTCASE_ID": { "type": "text" },
   "JIRA_FIND_BY_AUTO": { "type": "text" }
   }
   }
   }
   }

2. My logstash configuration file is

   input { file{ path => "C:/elkstack/elasticsearch-6.5.1/logs/git_commits.csv"
   start_position => "beginning"
   sincedb_path => "NUL" }
   }

   filter { csv { columns => [ "GIT_ORG",
   "GIT_REPOS",
   "COMMIT_SHA1",
   "COMMIT_AUTHOR",
   "COMMIT_DATE",
   "COMMIT_COMMENT",
   "JIRA_KEY",
   "JIRA_ISSUE_TYPE",
   "JIRA_ISSUE_CATEGORY",
   "JIRA_PRIORITY",
   "JIRA_COMPONENTS",
   "JIRA_DEFECT_TYPE",
   "JIRA_TESTCASE_ID",
   "JIRA_FIND_BY_AUTO"]
   separator => ","}

    }
   

   output {
   elasticsearch {
   action => "index"
   hosts => "localhost:9200"
   index => "logstash-git"
   manage_template => true
   template => "C:/elkstack/gittemp.json"
   template_name=> "gittemp"
   template_overwrite => true
   document_type => "doc"}

   stdout { codec => rubydebug }
   }

When I start logstash, there is below error in Elasticsearch, looks like there's more than 1 template matches the index logstash-git. But I check all templates that only 1 template matched by http://localhost:9200/_template?pretty and don't what where '_doc' comes from.

Please help.

[yaauie (Ry Biesemeyer)]

• _doc is the default document type in Elasticsearch, if an index is created without one. In Elasticsearch 6.x, an index must have exactly one document type (this is a part of the transition to document types being removed entirely from Elasticsearch)
• a mapping template is used only when a new index is created; if it is not in place before the index is created, then the template has no effect when added. it's useful when you want to have dynamic indices (such as indec-per-day pattern).
• you apparently already have an index logstash-git that has a _doc as its document type
• I would advise updating your template to be _doc, and remove the explicit document_type directive from the pipeline's elasticsearch output, as it is no longer necessary.

[cheriemilk]

Hi yaauie,

but I have a template already mapped and from the log of starting logstash, the template already detected. In the template, I clearly defined the document type is 'doc'. why it will still go to find the document type from other unmatched template? The logic seems problematic.

Thanks,
Cherie