Configure types and analyzed

Elastic Stack Logstash
#1

Hello, I'm new to the Elastic Stack and can't understand how to configure field types & whether strings should be analyzed or not.

I've found documentation that says I should use the Elasticsearch API but the indexes are only there after logs have started getting indexed and then it seems that configuration cannot be changed.

So I guess the way I should go are templates but I can't figure out how to configure those.

I'm using Filebeat to forward logs to Logstash which extracts fields and in turn passes them over to Elasticsearch.

#2

Hi,

The "Getting Started" chapter of the book "Elasticsearch - The Definitive Guide" is a good place to start:
https://www.elastic.co/guide/en/elasticsearch/guide/current/getting-started.html

It takes some time to read and it is not fully updated with latest changes but it's a good time investment if you plan to use Elasticsearch.

#3

Well that doesn't look like a very helpful response...

Anyway here's some more info on what I'm currently trying to do: as I understand it Logstash seems to be able to do the template management with these options.

So I created a simple template JSON file like:

{
  "template_name": {
    "template": "filebeat-*",
    "mappings": {
      "log_type": {
        "properties": {
          "field1": {
            "type": "string",
            "index": "not_analyzed"
          },
          ...
        }
      }
    }
  }
}

and setup my Elasticsearch output plugin on Logstash with these options:

manage_template => false
template_name => "template_name"
template => "/etc/logstash/template.json"
template_overwrite => true

and finally restarted Logstash AND deleted the daily indexes on Elasticsearch.

Doesn't look like it's working though. It seems not even the template is present on Elasticsearch as curl -XGET http://localhost:9200/_template only retuns {}.

EDIT: I manually uploaded the template JSON file to Elasticsearch (removing the first level of wrapping the settings with the template name) and it all works like a charm. Fields are typed correctly etc. So the only thing I seem to be missing is how to make Logstash do that work.

#4

That is wrong, it needs to be true.

While we are here to help we aren't here to hand you all the answers on a silver platter, most of us here (even Elastic team members) spend their own time here, ie we are volunteers.
If you don't get an answer you expect, perhaps rephrasing it rather than being snarky would help us help you :slight_smile:

#5

Thank for both quick replies honestly.

I didn't mean to be mean but just wanted to point out that a suggestion for the Getting Started guide in response a specific question is not useful and it can be hurtful in case people that want to help are looking for unanswered questions.

Anyway it looks like manage_template => true did the trick :slight_smile: Thanks!

#6