# Logstash default document type from logs to doc

**URL:** <https://discuss.elastic.co/t/logstash-default-document-type-from-logs-to-doc/217998>\
**Category:** Logstash\
**Created:** [February 5, 2020, 1:36pm UTC](https://discuss.elastic.co/t/logstash-default-document-type-from-logs-to-doc/217998 "2020-02-05T13:36:39Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![alessandro](https://avatars.discourse-cdn.com/v4/letter/a/ac91a4/32.png) [@alessandro](https://discuss.elastic.co/u/alessandro)\
**Post date:** [February 5, 2020, 1:36pm UTC](https://discuss.elastic.co/t/logstash-default-document-type-from-logs-to-doc/217998/1 "2020-02-05T13:36:39Z")

</div>

Hi forum,  
I would like to ask an advice for a smooth transition from Logstash version 5.6.4 to 6.8.6, given that I have recently updated succesfully both Elasticsearch and Kibana.

My main doubts are about the breaking change of the document type field, that was by default _logs_ in the former and then _doc_ in the latter version mentioned above, for consistency with Beats.

I am aware, reading other posts on the same topic, that I can use the mutate statement to make the change within the _filter_ section as follows:

```auto
filter{
        ...
        mutate {
          replace => { "type" => "doc" }
        }
}

```

But I would like to check if there is a strategy to perform this operation is a smooth way, avoiding data loss, since I am creating daily indexes and ideally I would like to start the new day with the correct type across all indexes.

Therefore I cannot apply the mutate statement without embedding it in a statement that for example compares the timestamp and applies the modification only after midnight, to avoid indexing issues due to the multiple conflicting type _[doc, logs]_ within the same index.

Does anybody have any idea how this problem could be tackled?

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\
**Post date:** [February 5, 2020, 11:07pm UTC](https://discuss.elastic.co/t/logstash-default-document-type-from-logs-to-doc/217998/2 "2020-02-05T23:07:36Z")

</div>

The Elasticsearch output plugin will detect the version of Elasticsearch it is talking to, and attempt to do the right thing:

- If it is talking to Elasticsearch 8+ where document types are removed, it will not send a document type at all (even if it is explicitly asked for with `document_type`); else
- if `document_type` is specified, it is used; else
- if it is talking to Elasticsearch 7.x, it will send `_doc` as the document type; else
- if it is talking to Elasticsearch 6.x, it will send `doc` as the document type; else
- it the event has a `type` fields, it will be used as the document type; else
- it will fall back to `doc` if the `type` is not specified.

Sources:

- [https://github.com/logstash-plugins/logstash-output-elasticsearch/blob/v10.3.1/lib/logstash/outputs/elasticsearch/common.rb#L67-L69](https://github.com/logstash-plugins/logstash-output-elasticsearch/blob/v10.3.1/lib/logstash/outputs/elasticsearch/common.rb#L67-L69)
- [https://github.com/logstash-plugins/logstash-output-elasticsearch/blob/v10.3.1/lib/logstash/outputs/elasticsearch/common.rb#L81](https://github.com/logstash-plugins/logstash-output-elasticsearch/blob/v10.3.1/lib/logstash/outputs/elasticsearch/common.rb#L81)

---

<div class="post-metadata">

**Author:** ![alessandro](https://avatars.discourse-cdn.com/v4/letter/a/ac91a4/32.png) [@alessandro](https://discuss.elastic.co/u/alessandro)\
**Post date:** [February 6, 2020, 3:26pm UTC](https://discuss.elastic.co/t/logstash-default-document-type-from-logs-to-doc/217998/3 "2020-02-06T15:26:08Z")

</div>

Thanks for the reply Ry,  
my question was more related to how to guarantee a smooth transition if I make the upgrade at some point in time say _x_.

I would like to avoid the condition where messages indexed before _x_ have _logs_ as type and messages indexed after _x_ have type _doc_.

This is how I was thinking to modify the logstash configuration file, given that _x_ is some point in time on 6th February and new indexes are created at night when @timestamp hour is 01:00

```auto
filter{
...

  if [@metadata][type] == "system_logs"
  {
    mutate{
      # enable timestamp comparison adding a new field
      add_field => { "str_dt_new" => "2020-02-07T01:00:00.000Z"}
    }
    date {
      match => ["str_dt_new", "YYYY-MM-dd'T'HH:mm:ss.SSSZ"]
      target => "constant_date"
    }
  }
  if [@timestamp] <= [constant_date] {
    mutate {
      # for consistency with messages sent before changing config file, in the 
      # same day (i.e. same ES index)
      replace => { "type" => "logs" }
    }
  }
  else {
    mutate {
      # required from LS v6.x
      replace => { "type" => "doc" }
    }
  }

}

```

Does it make sense to you?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 5, 2020, 3:26pm UTC](https://discuss.elastic.co/t/logstash-default-document-type-from-logs-to-doc/217998/4 "2020-03-05T15:26:12Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.