Document_type deprecated?

Elastic Stack Beats
1

Upgrading from 5.4 to 5.5, I see that document_type in Filebeat is deprecated because _type is going away in Elasticsearch 6.0.

Not a problem, as I'm not using multiple types per index.

But what do I do right now? - my natural inclination is to delete anything deprecated as soon as it becomes deprecated so I don't have anything breaking later, when it finally goes away. But I've got "document_type: wombat" in my filebeat.yml, and a matching

"mappings": { "wombat": {

in my template definition. Won't the template stop working if I simply delete the document_type?

2

I think that you will need to declare document_type as a custom field under fields, this way the type field will take the place of the _type field, as stated on this link.

I use the type field to allow me filter to in logstash, to use it with Filebeat 5.5 I've just declared it as a custom field under fields, as the example bellow

filebeat:
 prospectors:
  - input_type: log
    paths:
     - C:\inetpub\logs\LogFiles\W3SVC1\*.log
    fields:
      document_type: my-type
    exclude_lines: ["^#"]
    exclude_files: [".zip"]
    fields_under_root: true
    ignore_older: 24h
1 Like
3

I think that doesn't answer my question. I don't want or need multiple types, I'm not using them. What I've asked is what would happen to the template if I removed document_type (obviously replacing it with a custom field of my own would be replacing it with a field that the template knew nothing about).

4

document_type is not used for _type field. Beats (since 5.5) default _type to docs. This behaviour can not be overwritten. Still document_type sets the type field.

1 Like
5

That's not what I see. I've now got 5.5.1 running and I'm seeing "_type" still set to the value of "document_type".

6

Hm.... checking the 5.5 branch I clearly see _type being hardcoded to docs.

The _type field is set here: https://github.com/elastic/beats/blob/5.5/libbeat/outputs/elasticsearch/client.go#L325

with eventType being hardcoded to "doc": https://github.com/elastic/beats/blob/5.5/libbeat/outputs/elasticsearch/client.go#L96

Are you using the Elasticsearch output or Logstash?

7

Sorry, I misread you - I thought you'd said the changed behaviour was in Elasticsearch 5.5 but you said it was in Beats 5.5. Just right now I'm running Elasticsearch 5.5, Logstash 5.5, but Filebeat still at 5.4 (until I've done some fiddling with the Ansible scripts that deploys it).

8

Yep, it's in filebeat.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.