My filebeat client is sending data records with a numeric field name of 'accountID' to my elasticsearch server. When I create a (Kibana) visualization with this data, my graphs show the numberic 'accountID' values on the X-axis as expected. My goal is to map these numeric accountID values to an actual customer name. (Without the name, an accountID lookup in the database has to be executed, inefficient at best...) My failed solution:
Inner Objects - Use the elasticsearch mechanism called Inner Objects by adding the following values to the 'filebeat-index-template.json
{
"accountID": [
{ "ACCOUNT_ID": "1674", "SHORT_NAME": "Hickory"},
{ "ACCOUNT_ID": "887", "SHORT_NAME": "Dickory"},
{ "ACCOUNT_ID": "862", "SHORT_NAME": "Doc"},
]
}
}
Any suggestions for this novice to display a customer name, rather than my vague account_id??
Magnus
Thanks to your input, I installed the logstash-filter-translate plugin on my ELK server.
Based on what I read, I needed to configure the dictionary. I assume that the config file for this was located in /root/logstash-filter-translate-master/spec/filters/translate_spec.rb
I added these lines:
describe "Account_id translation" do
let(:config) do
{
"field" => "accountID",
"destination" => "translation",
"dictionary" => [ "1674", "Hickory",
"887", "Dickory",
"862", "Doc",
"1804", "Misc" ],
"exact" => true,
"regex" => false
}
end
let(:event) { LogStash::Event.new("status" => 200) }
it "return the exact translation" do
subject.register
subject.filter(event)
expect(event["translation"]).to eq("OK")
end
end
I restarted logstash with
service logstash restart
I expected to see the translated accountID values in Kibana, but did not.
Um, that's the tests for the plugin's source code. Editing it will get you nowhere. Use the plugin's dictionary_path option to select which file contains the mappings you want to make.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.