Document_type deprecated?


(Tim Ward) #1

Upgrading from 5.4 to 5.5, I see that document_type in Filebeat is deprecated because _type is going away in Elasticsearch 6.0.

Not a problem, as I'm not using multiple types per index.

But what do I do right now? - my natural inclination is to delete anything deprecated as soon as it becomes deprecated so I don't have anything breaking later, when it finally goes away. But I've got "document_type: wombat" in my filebeat.yml, and a matching

"mappings": { "wombat": {

in my template definition. Won't the template stop working if I simply delete the document_type?


(Leandro Maciel) #2

I think that you will need to declare document_type as a custom field under fields, this way the type field will take the place of the _type field, as stated on this link.

I use the type field to allow me filter to in logstash, to use it with Filebeat 5.5 I've just declared it as a custom field under fields, as the example bellow

filebeat:
 prospectors:
  - input_type: log
    paths:
     - C:\inetpub\logs\LogFiles\W3SVC1\*.log
    fields:
      document_type: my-type
    exclude_lines: ["^#"]
    exclude_files: [".zip"]
    fields_under_root: true
    ignore_older: 24h

Recommended way of telling Logstash type of logs
(Tim Ward) #3

I think that doesn't answer my question. I don't want or need multiple types, I'm not using them. What I've asked is what would happen to the template if I removed document_type (obviously replacing it with a custom field of my own would be replacing it with a field that the template knew nothing about).


(Steffen Siering) #4

document_type is not used for _type field. Beats (since 5.5) default _type to docs. This behaviour can not be overwritten. Still document_type sets the type field.


(Tim Ward) #5

That's not what I see. I've now got 5.5.1 running and I'm seeing "_type" still set to the value of "document_type".


(Steffen Siering) #6

Hm.... checking the 5.5 branch I clearly see _type being hardcoded to docs.

The _type field is set here: https://github.com/elastic/beats/blob/5.5/libbeat/outputs/elasticsearch/client.go#L325

with eventType being hardcoded to "doc": https://github.com/elastic/beats/blob/5.5/libbeat/outputs/elasticsearch/client.go#L96

Are you using the Elasticsearch output or Logstash?


Confusions around ELK6 types
(Tim Ward) #7

Sorry, I misread you - I thought you'd said the changed behaviour was in Elasticsearch 5.5 but you said it was in Beats 5.5. Just right now I'm running Elasticsearch 5.5, Logstash 5.5, but Filebeat still at 5.4 (until I've done some fiddling with the Ansible scripts that deploys it).


(Steffen Siering) #8

Yep, it's in filebeat.


(system) #9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.