# Can i convert latin-1 (ISO8859\_1) into UTF-8 available?

**URL:** https://discuss.elastic.co/t/can-i-convert-latin-1-iso8859-1-into-utf-8-available/96898
**Category:** Logstash
**Created:** [August 14, 2017, 3:32am UTC](https://discuss.elastic.co/t/can-i-convert-latin-1-iso8859-1-into-utf-8-available/96898 "2017-08-14T03:32:25Z")
**Posts on this page:** 12
**Page:** 1

<div class="post-metadata">

### Author: ![Prapat\_Polchan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prapat_polchan/32/20060_2.png) [@Prapat\_Polchan](https://discuss.elastic.co/u/Prapat_Polchan)
#### Post date: [August 14, 2017, 3:32am UTC](https://discuss.elastic.co/t/can-i-convert-latin-1-iso8859-1-into-utf-8-available/96898/1 "2017-08-14T03:32:25Z")

</div>

i using JDBC input on logstash but can't read latin-1 into elasticsearch anyone can convert guide for me ?

this result problem.  
 ![Selection_124](https://us1.discourse-cdn.com/elastic/original/3X/0/7/07cf754f9263064958516ce77f259f4c5efe3c3b.png)

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 14, 2017, 5:29am UTC](https://discuss.elastic.co/t/can-i-convert-latin-1-iso8859-1-into-utf-8-available/96898/2 "2017-08-14T05:29:37Z")

</div>

Have you looked at the jdbc input's `columns_charset` option?

---

<div class="post-metadata">

### Author: ![Prapat\_Polchan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prapat_polchan/32/20060_2.png) [@Prapat\_Polchan](https://discuss.elastic.co/u/Prapat_Polchan)
#### Post date: [August 14, 2017, 5:58am UTC](https://discuss.elastic.co/t/can-i-convert-latin-1-iso8859-1-into-utf-8-available/96898/3 "2017-08-14T05:58:41Z")

</div>

Sure , but not work

if me convert in jsp

i used this code

new String(rs3.getString("custstatusname").getBytes("ISO8859\_1"), "TIS-620")

but i no idea on logstash

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 14, 2017, 6:38am UTC](https://discuss.elastic.co/t/can-i-convert-latin-1-iso8859-1-into-utf-8-available/96898/4 "2017-08-14T06:38:01Z")

</div>

> Sure , but not work

Please show the configuration you tried and the result you get. Use a `stdout { codec => rubydebug }` output.

---

<div class="post-metadata">

### Author: ![Prapat\_Polchan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prapat_polchan/32/20060_2.png) [@Prapat\_Polchan](https://discuss.elastic.co/u/Prapat_Polchan)
#### Post date: [August 14, 2017, 6:46am UTC](https://discuss.elastic.co/t/can-i-convert-latin-1-iso8859-1-into-utf-8-available/96898/5 "2017-08-14T06:46:00Z")

</div>

Result

![Selection_127](https://us1.discourse-cdn.com/elastic/original/3X/0/a/0a1f4a8b912b453bcf1ceb33cf457fc01d5e9c53.png)

![Selection_126](https://us1.discourse-cdn.com/elastic/original/3X/8/1/8192e4ac6d8ced3eb671845827ab02f0c41e4b42.png)

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 14, 2017, 7:42am UTC](https://discuss.elastic.co/t/can-i-convert-latin-1-iso8859-1-into-utf-8-available/96898/6 "2017-08-14T07:42:38Z")

</div>

Don't post screenshots if you can use copy/paste.

Well, that configuration looks correct. Are you sure the columns really are ISO8859-1? What is the `mrr_lname` column supposed to contain? The UTF-8 string "Ë§ÉìÈĀÕ" contains the following octets:

```
c3 8b c2 a7 c3 89 c3 ac c3 88 c4 80 c3 95

```

This isn't even valid ISO8859-1 since 80, 88, 89, and 8b aren't defined in that character set. It's possible that the UTF-8 string you ended up with can't be translated back to ISO8859-1 like I just did but it still looks fishy to me.

---

<div class="post-metadata">

### Author: ![Prapat\_Polchan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prapat_polchan/32/20060_2.png) [@Prapat\_Polchan](https://discuss.elastic.co/u/Prapat_Polchan)
#### Post date: [August 16, 2017, 6:43am UTC](https://discuss.elastic.co/t/can-i-convert-latin-1-iso8859-1-into-utf-8-available/96898/7 "2017-08-16T06:43:35Z")

</div>

i wrote this

> filter {  
> ruby {  
> init =\>"require 'net/https'"  
> code =\> 'event.set("cust\_issby", event.get("cust\_issby").force\_encoding("ISO-8859-1").encode("TIS-620"))'  
> }  
> }

but not work.... my charset need TIS-620 UTF-8 can't used this case

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 16, 2017, 8:02am UTC](https://discuss.elastic.co/t/can-i-convert-latin-1-iso8859-1-into-utf-8-available/96898/8 "2017-08-16T08:02:57Z")

</div>

What do you mean? Is the data from the database actually encoded in TIS-620?

---

<div class="post-metadata">

### Author: ![Prapat\_Polchan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prapat_polchan/32/20060_2.png) [@Prapat\_Polchan](https://discuss.elastic.co/u/Prapat_Polchan)
#### Post date: [August 16, 2017, 8:14am UTC](https://discuss.elastic.co/t/can-i-convert-latin-1-iso8859-1-into-utf-8-available/96898/9 "2017-08-16T08:14:04Z")

</div>

my database default ISO8859-1 but output language need convert to TIS-620 Only

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 16, 2017, 8:45am UTC](https://discuss.elastic.co/t/can-i-convert-latin-1-iso8859-1-into-utf-8-available/96898/10 "2017-08-16T08:45:58Z")

</div>

I don't understand how it could possibly make sense to convert from ISO8859-1 to TIS-620. The former character set doesn't contain any Thai characters so only the lower seven bits _can_ be converted into TIS-620.

Is the situation that the database stores TIS-620 text in columns that have been marked as ISO8859-1?

---

<div class="post-metadata">

### Author: ![Prapat\_Polchan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prapat_polchan/32/20060_2.png) [@Prapat\_Polchan](https://discuss.elastic.co/u/Prapat_Polchan)
#### Post date: [August 17, 2017, 7:14am UTC](https://discuss.elastic.co/t/can-i-convert-latin-1-iso8859-1-into-utf-8-available/96898/11 "2017-08-17T07:14:23Z")

</div>

Yaha , it work, i custom filter code ruby

this work:

```
filter {
  ruby {
  	init =>"require 'net/https' 
  			require 'iconv'"
  	code => "
	row = event.get('cust_issby').codepoints.to_a.pack('C*')
	row = Iconv.new('UTF-8','TIS-620').iconv(row)
  	event.set('cust_issby', row)
  	"
  }
}
```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 14, 2017, 7:14am UTC](https://discuss.elastic.co/t/can-i-convert-latin-1-iso8859-1-into-utf-8-available/96898/12 "2017-09-14T07:14:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
