# Custom index with field from xpath **URL:** \ **Category:** Logstash\ **Created:** [April 25, 2017, 2:16am UTC](https://discuss.elastic.co/t/custom-index-with-field-from-xpath/83478 "2017-04-25T02:16:24Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![daian.martinho](https://avatars.discourse-cdn.com/v4/letter/d/8c91f0/32.png) [@daian.martinho](https://discuss.elastic.co/u/daian.martinho)\ **Post date:** [April 25, 2017, 2:16am UTC](https://discuss.elastic.co/t/custom-index-with-field-from-xpath/83478/1 "2017-04-25T02:16:24Z")
Filebeat is sending xml files to logstash. ``` input { beats { port => "5044" } } filter { ## interpret the message as XML if [type] == "nessus-report" { xml { source => "message" store_xml => false xpath =>[ "/NessusClientData_v2/Report/@name","report_name", "/NessusClientData_v2/Report/ReportHost","report_host" ] } split { field => "report_host" } xml { source => "report_host" store_xml => false xpath =>[ "/ReportHost/ReportItem","report_item", "/ReportHost/@name","report_host_name", "/ReportHost/HostProperties/tag[@name='HOST_START']/text()","report_host_start", "/ReportHost/HostProperties/tag[@name='HOST_END']/text()","report_host_end" ] } split { field => "report_item" } xml { source => "report_item" store_xml => false xpath => [ "/ReportItem/@port","report_item_port" ] } mutate { remove_field => ["message","report_host","report_item"] replace => { "report_host_start" => "%{report_host_start[0]}" } replace => { "report_host_end" => "%{report_host_end[0]}" } convert => { "report_item_severity" => "integer" } } date { match => ["report_host_start", "EEE MMM dd HH:mm:ss yyyy"] target => "report_host_start" locale => "en_US" } date { match => ["report_host_end", "EEE MMM dd HH:mm:ss yyyy"] target => "report_host_end" locale => "en_US" } } } output { if [type]=="nessus-report" { elasticsearch { hosts => ["10.99.40.16:9200"] manage_template => false index => "nessus-report-%{report_name}-%{+YYYY.MM.dd}" document_type => "nessus-report" } } } ``` I'm trying to insert field report\_name into index but the result is: ``` _index= nessus-report-2017-04-24 type = %{[@metadata][type]} report_name = correct-name ``` can someone help me? tnks !! ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 25, 2017, 7:19am UTC](https://discuss.elastic.co/t/custom-index-with-field-from-xpath/83478/2 "2017-04-25T07:19:10Z")
You mean the `type` of the resulting document is "%{[@metadata][type]}"? That's impossible with the configuration you posted. ---
**Author:** ![daian.martinho](https://avatars.discourse-cdn.com/v4/letter/d/8c91f0/32.png) [@daian.martinho](https://discuss.elastic.co/u/daian.martinho)\ **Post date:** [April 25, 2017, 1:29pm UTC](https://discuss.elastic.co/t/custom-index-with-field-from-xpath/83478/3 "2017-04-25T13:29:56Z")
Ok, but what about empty %{report\_name} at index? The sintax is correct? Thank you ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 25, 2017, 1:50pm UTC](https://discuss.elastic.co/t/custom-index-with-field-from-xpath/83478/4 "2017-04-25T13:50:49Z")
> Ok, but what about empty %{report\_name} at index? I'm not sure what you mean. The index name is "nessus-report-2017-04-24", but that string can't possibly have been produced by the pattern "nessus-report-%{report\_name}-%{+YYYY.MM.dd}". As I said, the result you claim you get is impossible with the configuration you've shown us. ---
**Author:** ![daian.martinho](https://avatars.discourse-cdn.com/v4/letter/d/8c91f0/32.png) [@daian.martinho](https://discuss.elastic.co/u/daian.martinho)\ **Post date:** [April 25, 2017, 2:31pm UTC](https://discuss.elastic.co/t/custom-index-with-field-from-xpath/83478/5 "2017-04-25T14:31:10Z")
> but that string can't possibly have been produced by the pattern "nessus-report-%{report\_name}-%{+YYYY.MM.dd}" but it is and i don't understand why. result doc example: report\_name: CTB-AdvNetworkScan index: nessus-report-2017-04-24 as you see, the field report\_name has a value. more then that, if it was an %{report\_name} problem, the index should be "nessus-report--2017-04-24" (with 1 more "-" between report and date) i'm trying to understand that this conf file is named nessus-xml.conf. i have another conf file with an output that doesn't have conditional dependency, named odin.conf. ``` elasticsearch { hosts => ["10.99.40.16:9200"] manage_template => false index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}" document_type => "%{[@metadata][type]}" } ``` can be that? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 25, 2017, 2:50pm UTC](https://discuss.elastic.co/t/custom-index-with-field-from-xpath/83478/6 "2017-04-25T14:50:22Z")
Yeah, that might be the culprit. Logstash sends all events to all outputs unless you wrap them in conditionals. ---
**Author:** ![daian.martinho](https://avatars.discourse-cdn.com/v4/letter/d/8c91f0/32.png) [@daian.martinho](https://discuss.elastic.co/u/daian.martinho)\ **Post date:** [April 25, 2017, 8:02pm UTC](https://discuss.elastic.co/t/custom-index-with-field-from-xpath/83478/7 "2017-04-25T20:02:06Z")
i think that was the problem. now i'm trying to lowercase and remove whitespaces from the report\_name field ``` mutate { add_field => { "index_report_name" => "%{report_name}" } lowercase => ["index_report_name"] gsub => [ #remove all whitespaces "index_report_name", " ", "" ] } ``` but that still not working ``` [2017-04-25T16:43:02,678][WARN][logstash.outputs.elasticsearch] Failed action. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"nessus-report-VA vsware-2017.04.25", :_type=>"nessus-report", :_routing=>nil}, 2017-04-25T19:41:54.197Z CSB-OdinProxy %{message}], :response=>{"index"=>{"_index"=>"nessus-report-VA vsware-2017.04.25", "_type"=>"nessus-report", "_id"=>nil, "status"=>400, "error"=>{"type"=>"invalid_index_name_exception", "reason"=>"Invalid index name [nessus-report-VA vsware-2017.04.25], must not contain the following characters [, \", *, \\, <, |, ,, >, /, ?]", "index_uuid"=>"_na_", "index"=>"nessus-report-VA vsware-2017.04.25"}}}} ``` ---
**Author:** ![daian.martinho](https://avatars.discourse-cdn.com/v4/letter/d/8c91f0/32.png) [@daian.martinho](https://discuss.elastic.co/u/daian.martinho)\ **Post date:** [April 25, 2017, 9:34pm UTC](https://discuss.elastic.co/t/custom-index-with-field-from-xpath/83478/8 "2017-04-25T21:34:24Z")
ok, i had to do this: ``` mutate { add_field => { "index_report_name" => "%{report_name}" } } mutate{ lowercase => ["index_report_name"] } mutate{ gsub => [ #remove all whitespaces "index_report_name", " ", "" ] } ``` now i have indexes like: nessus-report-a-2017.04.25 nessus-report-b-2017.04.25 nessus-report-c-2017.04.25 nessus-report-2017.04.25 (this one is index a + b + c. this is normal in elasticsearch?) ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 26, 2017, 5:08am UTC](https://discuss.elastic.co/t/custom-index-with-field-from-xpath/83478/9 "2017-04-26T05:08:46Z")
> nessus-report-2017.04.25 (this one is index a + b + c. this is normal in elasticsearch?) No. You must have an elasticsearch output somewhere that sets `index` to nessus-report-2017.04.25. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 24, 2017, 5:20am UTC](https://discuss.elastic.co/t/custom-index-with-field-from-xpath/83478/10 "2017-05-24T05:20:49Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.