Index creation based on file name


(Dilip Sharma) #1

Hi All
i want to create dynamic index based on the file name --
currently my input part is -

input {
file {
path => "E:\Local_Elasticsearch\logstashv5\datafiles/*.csv"
start_position => "beginning"

}
}

and my output part is -

output {
	elasticsearch {
		hosts => "localhost"
		index => "testindex1"
		document_type => "tesdata1"
	}
	stdout{}

}

how can i have index name based on file is uploading from data folder ..
any help or suggestion .

Thanks


(Dilip Sharma) #2

i used index => "logstash-%{+YYYYMMdd}" this method to solve my issue .
Thanks


(Paris Mermigkas) #3

What you've posted does not create indices based on filename, but rather daily indices.

If you still want filename-based indexing, you can use the "path" field of each event, like so

filter {
        ruby {
            code => "
                event.set('index_name',event.get('path').split('/')[-1].gsub('.csv',''))
            "
        }
}
output {
	elasticsearch {
		hosts => "localhost"
		index => "%{index_name}"
		document_type => "tesdata1"
	}
}

(Dilip Sharma) #4

yes , but that solved my problem, i m ok with daily index . but still thanks a lot for this solution i will use that somewhere .

thanks
Dilip


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.