Default value of the type field

snuffkin · February 9, 2018, 5:23pm

Elasticsearch Reference says "The preferred type name is _doc".
https://www.elastic.co/guide/en/elasticsearch/reference/current/removal-of-types.html#_schedule_for_removal_of_mapping_types

But, logstash-output-elasticsearch sets "doc" to the type field for default.

github.com

logstash-plugins/logstash-output-elasticsearch/blob/master/lib/logstash/outputs/elasticsearch/common.rb#L207


  end


  if @version_type
    params[:version_type] = event.sprintf(@version_type)
  end


  params
end


# Determine the correct value for the 'type' field for the given event
DEFAULT_EVENT_TYPE="doc".freeze
def get_event_type(event)
  # Set the 'type' value for the index.
  type = if @document_type
           event.sprintf(@document_type)
         else
           if client.maximum_seen_major_version < 6
             event.get("type") || DEFAULT_EVENT_TYPE
           else
             DEFAULT_EVENT_TYPE
           end

Should logstash-output-elasticsearch set "_doc" to the type field?

quentin.legraverend · February 13, 2018, 2:01pm

Hi snuffkin,

I was about to do the same post when I saw your's. So, same issue here. Why is the default value of "doc" used in logstash-output-elasticsearch ? (source: https://www.elastic.co/guide/en/logstash/6.2/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-document_type).

Moreover, that's been a while I removed all the document_type fields in my logstash configuration files but I still have this warning in my logfiles:

[2018-02-13T13:32:45,781][WARN ][logstash.outputs.elasticsearch] You are using a deprecated config setting "document_type" set in elasticsearch. Deprecated settings will continue to work, but are scheduled for removal from logstash in the future. Document types are being deprecated in Elasticsearch 6.0, and removed entirely in 7.0. You should avoid this feature If you have any questions about this, please visit the #logstash channel on freenode irc. blablablabla.....}

I use a 6.2.0 stack and the pipeline management feature of Kibana. I've absolutely no configuration in /etc/logstash and all the configurations stored in the pipeline management (in Elasticsearch) are "document_type"-free in Elasticsearch output.

Could we have any answer from the core-devs of Logstash ?

Have a nice day,
Kuaaaly

byoungb · February 17, 2018, 4:46am

Yeah I agree I just wasted way too much time tracking down an mapping type conflict issue caused by me not realizing that logstash was sending "doc" and my template index had defined "_doc" like there documentation recommended.

system · March 17, 2018, 4:47am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.