Can i convert latin-1 (ISO8859_1) into UTF-8 available?

Elastic Stack Logstash
1

i using JDBC input on logstash but can't read latin-1 into elasticsearch anyone can convert guide for me ?

this result problem.
Selection_124

2

Have you looked at the jdbc input's columns_charset option?

3

Sure , but not work

if me convert in jsp

i used this code

new String(rs3.getString("custstatusname").getBytes("ISO8859_1"), "TIS-620")

but i no idea on logstash

4

Sure , but not work

Please show the configuration you tried and the result you get. Use a stdout { codec => rubydebug } output.

5

Result

Selection_127

Selection_126

6

Don't post screenshots if you can use copy/paste.

Well, that configuration looks correct. Are you sure the columns really are ISO8859-1? What is the mrr_lname column supposed to contain? The UTF-8 string "˧ÉìÈĀÕ" contains the following octets:

c3 8b c2 a7 c3 89 c3 ac  c3 88 c4 80 c3 95

This isn't even valid ISO8859-1 since 80, 88, 89, and 8b aren't defined in that character set. It's possible that the UTF-8 string you ended up with can't be translated back to ISO8859-1 like I just did but it still looks fishy to me.

7

i wrote this

filter {
ruby {
init =>"require 'net/https'"
code => 'event.set("cust_issby", event.get("cust_issby").force_encoding("ISO-8859-1").encode("TIS-620"))'
}
}

but not work.... my charset need TIS-620 UTF-8 can't used this case

8

What do you mean? Is the data from the database actually encoded in TIS-620?

9

my database default ISO8859-1 but output language need convert to TIS-620 Only

10

I don't understand how it could possibly make sense to convert from ISO8859-1 to TIS-620. The former character set doesn't contain any Thai characters so only the lower seven bits can be converted into TIS-620.

Is the situation that the database stores TIS-620 text in columns that have been marked as ISO8859-1?

11

Yaha , it work, i custom filter code ruby

this work:

filter {
  ruby {
  	init =>"require 'net/https' 
  			require 'iconv'"
  	code => "
	row = event.get('cust_issby').codepoints.to_a.pack('C*')
	row = Iconv.new('UTF-8','TIS-620').iconv(row)
  	event.set('cust_issby', row)
  	"
  }
}
12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.