Need help for error " java.lang.IllegalArgumentException: Rejecting mapping update to [logstash-git] as the final mapping would have more than 1 type: [_doc, doc]"

Elastic Stack Logstash
1

Hi team - need your help for this painful issue.

  1. My ES Index template with index pattern is "logstash-git"

    {
    "template": "git_template",
    "index_patterns":["logstash-git"],
    "mappings": {
    "doc": {
    "properties": {
    "GIT_ORG": { "type": "text" },
    "GIT_REPOS": { "type": "text" },
    "COMMIT_SHA1": { "type": "text" },
    "COMMIT_AUTHOR": { "type": "text" },
    "COMMIT_DATE": { "type": "date",
    "format": "yyyy-MM-dd"},
    "COMMIT_COMMENT": { "type": "text" },
    "JIRA_KEY": { "type": "text" },
    "JIRA_ISSUE_TYPE": { "type": "text" },
    "JIRA_ISSUE_CATEGORY": { "type": "text" },
    "JIRA_PRIORITY": { "type": "text" },
    "JIRA_COMPONENTS": { "type": "text" },
    "JIRA_DEFECT_TYPE": { "type": "text" },
    "JIRA_TESTCASE_ID": { "type": "text" },
    "JIRA_FIND_BY_AUTO": { "type": "text" }
    }
    }
    }
    }

  2. My logstash configuration file is

    input { file{ path => "C:/elkstack/elasticsearch-6.5.1/logs/git_commits.csv"
    start_position => "beginning"
    sincedb_path => "NUL" }
    }

    filter { csv { columns => [ "GIT_ORG",
    "GIT_REPOS",
    "COMMIT_SHA1",
    "COMMIT_AUTHOR",
    "COMMIT_DATE",
    "COMMIT_COMMENT",
    "JIRA_KEY",
    "JIRA_ISSUE_TYPE",
    "JIRA_ISSUE_CATEGORY",
    "JIRA_PRIORITY",
    "JIRA_COMPONENTS",
    "JIRA_DEFECT_TYPE",
    "JIRA_TESTCASE_ID",
    "JIRA_FIND_BY_AUTO"]
    separator => ","}

     }

    output {
    elasticsearch {
    action => "index"
    hosts => "localhost:9200"
    index => "logstash-git"
    manage_template => true
    template => "C:/elkstack/gittemp.json"
    template_name=> "gittemp"
    template_overwrite => true
    document_type => "doc"}

    stdout { codec => rubydebug }
    }

When I start logstash, there is below error in Elasticsearch, looks like there's more than 1 template matches the index logstash-git. But I check all templates that only 1 template matched by http://localhost:9200/_template?pretty and don't what where '_doc' comes from.

Please help.

2
  • _doc is the default document type in Elasticsearch, if an index is created without one. In Elasticsearch 6.x, an index must have exactly one document type (this is a part of the transition to document types being removed entirely from Elasticsearch)
  • a mapping template is used only when a new index is created; if it is not in place before the index is created, then the template has no effect when added. it's useful when you want to have dynamic indices (such as indec-per-day pattern).
  • you apparently already have an index logstash-git that has a _doc as its document type
  • I would advise updating your template to be _doc, and remove the explicit document_type directive from the pipeline's elasticsearch output, as it is no longer necessary.
3

Hi yaauie,

but I have a template already mapped and from the log of starting logstash, the template already detected. In the template, I clearly defined the document type is 'doc'. why it will still go to find the document type from other unmatched template? The logic seems problematic.

Thanks,
Cherie