# Ingest-attachment can't recognize content type for text encoded in win-1251

**URL:** <https://discuss.elastic.co/t/ingest-attachment-cant-recognize-content-type-for-text-encoded-in-win-1251/183538>\
**Category:** Elasticsearch\
**Created:** [May 30, 2019, 1:23pm UTC](https://discuss.elastic.co/t/ingest-attachment-cant-recognize-content-type-for-text-encoded-in-win-1251/183538 "2019-05-30T13:23:25Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![aledyad](https://avatars.discourse-cdn.com/v4/letter/a/90db22/32.png) [@aledyad](https://discuss.elastic.co/u/aledyad)\
**Post date:** [May 30, 2019, 1:23pm UTC](https://discuss.elastic.co/t/ingest-attachment-cant-recognize-content-type-for-text-encoded-in-win-1251/183538/1 "2019-05-30T13:23:25Z")

</div>

Hi.

Use ES 6.7.2 with ingest-attachment plugin.  
I have text files (\*.txt) in win-1251 encoding with russian chars. After put binary data to index, documents in index have no source, because ingest processor dont recognize content. If there is no russian chars - everything is ok.

```
POST _ingest/pipeline/_simulate
{
  "pipeline": {
    "processors": [
      {
      "attachment": {
        "field": "data",
        "target_field": "attachment",
        "properties": [
          "content",
          "content_length",
          "content_type",
          "language"
        ],
        "indexed_chars": -1
      }
      }
    ]
  },
  "docs": [
    {
      "_source": {
        "data": "8PPx"
      }
    }
  ]
}

```

returns

```
{
  "docs" : [
    {
      "doc" : {
        "_index" : "_index",
        "_type" : "_type",
        "_id" : "_id",
        "_source" : {
          "data" : "8PPx",
          "attachment" : {
            "content_type" : "application/octet-stream",
            "content_length" : 0
          }
        },
        "_ingest" : {
          "timestamp" : "2019-05-30T13:08:18.581Z"
        }
      }
    }
  ]
}

```

> "data": "8PPx"  
> 8PPx - this is text "рус" encoded in base64.

I tested recognition of text file via [tika-app.jar](https://tika.apache.org/1.5/gettingstarted.html) command line - encoding detects right: "Content-Type: text/plain; charset=windows-1251".

Why ingest-attachment not detect content type as "text/plain; charset=windows-1251"?

upd: for more complex text are the same "8uXq8fIg4iDq7uTo8O7i6uUgMTI1MQ" -\> "текст в кодировке 1251").

---

<div class="post-metadata">

**Author:** ![aledyad](https://avatars.discourse-cdn.com/v4/letter/a/90db22/32.png) [@aledyad](https://discuss.elastic.co/u/aledyad)\
**Post date:** [June 5, 2019, 12:15pm UTC](https://discuss.elastic.co/t/ingest-attachment-cant-recognize-content-type-for-text-encoded-in-win-1251/183538/2 "2019-06-05T12:15:30Z")

</div>

It seems the problem in tiko. It's dont recognize non-US ASCII chars as text without hints like file extension.

```
public boolean isMostlyAscii() {
    int control = count(0, 0x20);
    int ascii = count(0x20, 128);
    int safe = countSafeControl();
    return total > 0
            && (control - safe) * 100 < total * 2
            && (ascii + safe) * 100 > total * 90;
}

```

But national ascii chars has codes 0xc0-0xff.

If in test application specify file without extension, it recognize content like "application/octet-stream".  
I didn't found any settings, which can help me. ☹ May be try to fix tica-core.jar.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 3, 2019, 12:15pm UTC](https://discuss.elastic.co/t/ingest-attachment-cant-recognize-content-type-for-text-encoded-in-win-1251/183538/3 "2019-07-03T12:15:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.