Logstash encoding (previously in Kibana encoding)

Elastic Stack Logstash
1

Is it possible to change the charset that Kibana reports in? I am having issues where Kibana reports the replacement character when it is supposed to be an oslash. I have changed the encoding charset in both the input file and the elasticsearch output charset but neither of them had an effect on how Kibana displays them. Any tips?

Thank you!

2

Hi Colton,

Not sure about your question at all? Can you please tell us which version of Kibana/ES you are using? what do you mean by charset that Kibana reports in? is it on reporting or on the UI and a screenshot?

Thanks,
Bhavya

3

I am using the latest versions of Kibana/ES (5.4.1). Perhaps charset is the wrong way of saying it. There was a setting logstash where I could change the encoding when the file is read in:

file{
	path => "/opt/ELK/data/**/dbms*.prn"
	start_position => beginning
	sincedb_path => "/dev/null"
	ignore_older => 0
	type => "prn"
	codec => plain{charset => "ANSI_X3.4-1968"}
}

I also have this line of code at the output:

	if [type] == "prn" {
		elasticsearch{
			index => "prn-"
			document_type => "prnformat"
			template => "/opt/ELK/logstash/template/prn_template.json"
			template_name => "prn-*"
			template_overwrite => "true"
			codec => plain{charset => "ANSI_X3.4-1968"}
		}
	}

But in Kibana it reports the wrong character:

Here is what the character that is misrepresented in Kibana looks like in Notepad++

4

Ah gotcha. Thank you. Will hunt down someone who knows more about this.

Cheers,
Bhavya

5

Did you set these codecs because of problems without the setting?

What browser are you using? Did you try another browser if you have another one installed?

Can you try just doing a query in your browser address bar like this?

(You'll have to change some of this. Replace makelogs-0/apache with your index name and type.
and replace the query extension.raw=jpg with some query that gets the data you're looking for.

http://localhost:9200/makelogs-0/apache/_search?q=extension.raw=jpg

If the characters don't show up properly there, then it's not a Kibana issue. It's either an issue with the data loading or your OS or browser settings.

6

I set the codecs because when the files were being read in logstash they blew up the command window with warnings about the wrong encoding. When I added those lines the warnings went away but these strange characters stayed.

I was using Chrome. Here is the strange character in Internet Explorer:

Here is what a query found:

Do you know of what I should check next?

7

Hi Colton,

Just as a test, does this look like the correct Nordic O slash character in your Chrome and IE browser? Ø

screenshot:

That's from this page; http://sites.psu.edu/symbolcodes/windows/codealt/

If it looks correct, then your problem must be in how the data is loaded into Elasticsearch. I think you should create a new post in the Logstash forum.

If it looks incorrect, then I think it is a setting on your computer itself?

Lee

8

Yep. That looks correct. I'll move this post over to the Logstash forum

9

Colton,
Here's one more test I just did on my Kibana. I posted a doc with that char in it to a new index, and then did a search on that index and Kibana shows it correctly in the console. If I also create an index pattern I'm sure I would see it correctly in the rest of Kibana.

image.png950×475 33.5 KB

10

I have no knowledge how to use the Console feature in Kibana. So I am unsure how to run that test you did.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.