How can I set elasticsearch type in logstash conf file when importing?

Elastic Stack Logstash
1

I'm trying to import CSV file to elasticsearch through logstash.
data can be imported and type name is logs.
Is there any way to change type name?
Any setting to type name in logstash conf file when importing??

2

Are you talking about the event's document type or the field's mapping type?

3

i think field's document type.
it automatically names as 'logs' when I import data through logstash.

my conf file is as below

input {
  file {
    path => "/tmp/es_data_ori.csv"
    start_position => "beginning"
  }
}
filter {
  csv {
    columns => ["column1","column2","column3"]
    separator => ";"
  }
}
output {
  elasticsearch {
    hosts => ["192.168.33.10"]
    index => "test"
    type  => "items"  //any property to set type name like this??
  }
}

I'm just beginning to use ES and logstash.
I apologize in advance if I confuse any.

4

Add

type => "items"

to your file input or set

document_type => "items"

in your elasticsearch output. I prefer the former way since it gives you a type field that you can use to filter events from different sources in different ways.

5

i tried the setting that you mentioned above.
suddenly, logstash doesn't run itself.
according to log, logstash can't find /etc/logstash/conf.d.

i didn't change anything except the setting .
as I returned the setting yet logstash doesnt run...
what happend??

Is this irrelevant to this page issue???

6

according to log, logstash can't find /etc/logstash/conf.d.

Please show the exact error message.

7

/var/log/logstash/logstash.err

{:timestamp=>"2016-11-29T02:40:01.820000+0000", :message=>"No config files found: /etc/logstash/conf.d/*\nCan you make sure this path is a logstash config file?", :level=>:error}

i checked the directory /etc/logstash/conf.d and there is no file in it.
I haven't changed or removed anything.

8

I'm confused. What are you asking?

  • Logstash never deletes configuration files.
  • Logstash typically expects configuration files to reside in /etc/logstash/conf.d (and it obviously does in your case).
  • It's up to you to store the configuration files in /etc/logstash/conf.d.
9

sorry to make you confused.

I thought conf file was deleted or moved by logstash process but it was wrong.
just ignore these lines.

i checked the directory /etc/logstash/conf.d and there is no file in it.
I haven't changed or removed anything.

my question is simply why is logstash not able to start after trying to imprt CSV file?
If you have heard similar issue before, any suggestion?

10

my question is simply why is logstash not able to start after trying to imprt CSV file?

Have you stored your configuration file in /etc/logstash/conf.d? If yes, what's the new error message you get in the log?

11

I put empty conf file named conf.d as test in /etc/logstash/conf.d/ directory and error message is like this.

message=>"fetched an invalid config", :config=>"\n", :reason=>"Expected one of #, input, filter, output at line 2, column 1 (byte 2) after \n", :level=>:error}
12

Put the configuration file with the CSV import stuff in a file in /etc/logstash/conf.d. Over and out.

13

OK.
Thanks.

14

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.