Field "document_type" sent via filebeat as "type" field to ES is ananlyzed

Elastic Stack Kibana
1

I am using Filebeat 5.3-->Logstash 5.3-->Elasticserach 5.3.

While sending my documents using filebeat I am using document_type to tag a log file type as below :-1:

  • input_type: log
    paths:
    • /my/log/location/server.log
      document_type: jboss-server-logs

This document_type goes into elastic search as filed "type".

This filed "type" is searchable but analyzed. If I try to use the query in Kibana as "type : jboss-server-logs " it gives me all different logs files which have "jboss-server-logs", "jboss","logs" and each component broken.

I wanted to use this filed to particularly see at different log files I am pushing into one index.

How can i get rid of this filed being analyzed and make it a raw field?

2

Hey,

if you haven't changed the default mapping, you could filter on type.keyword:jboss-server-logs, which will be the unanalyzed version. If that field doesn't exist, you would need to change the mapping, to store the type as a keyword (sub)field: https://www.elastic.co/guide/en/elasticsearch/reference/5.3/keyword.html

Cheers,
Tim

3

Hey Tim,

Thanks for responding,

I do have the filed type.keyword available associated with this field.

I do also see another filed named "_type" which I can search on and that gives me intended results but as I need to transition the usage of Kibana to other teams, I wanted to keep it something simpler e.g. rather than using _type or type.keyword, if they can use type: <> that would make pretty clean and easy.

Is there a way I can have retrieve this field un-analyzed in Elastic Search with the same name as type?

Thanks,
VJ

4

Hey VJ,

you can't use the same name to retrieve the field unanalyzed and analyzed in the same way.
But you could change your mapping, so that the type field would be of type keyword, since I don't think you will ever need the analyzed version of this field, since you said they are just ids.

The _type field is actually something internal (thus the underscore) and you shouldn't use this. This might currently work, since apparently you use the type value also as an "ES document type", but that is already deprecated and will be removed in the upcoming versions.

That way you would only have a type field and it does what you want :slight_smile:
(If you still need the analyzed field, you could of course also think in renaming that to type.analyzed or similar.)

Cheers,
Tim

5

This mapping is something default which happens internally when the file beat specified field "document_type" is retrieved by logastash and ES.

Is it possible for me to change that mapping?

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.