# Erreur d'importation avec logstash sous linux

**URL:** https://discuss.elastic.co/t/erreur-dimportation-avec-logstash-sous-linux/233427
**Category:** Logstash
**Created:** [May 20, 2020, 12:28am UTC](https://discuss.elastic.co/t/erreur-dimportation-avec-logstash-sous-linux/233427 "2020-05-20T00:28:12Z")
**Posts on this page:** 20
**Page:** 1

<div class="post-metadata">

### Author: ![Katia](https://avatars.discourse-cdn.com/v4/letter/k/b5e925/32.png) [@Katia](https://discuss.elastic.co/u/Katia)
#### Post date: [May 20, 2020, 12:28am UTC](https://discuss.elastic.co/t/erreur-dimportation-avec-logstash-sous-linux/233427/1 "2020-05-20T00:28:12Z")

</div>

Bonjour,  
J'importe mes logs avec logstash vers elasticserach, ça marche bien en local ( sous windows).  
Mais pas sous linux, j'ai pleins d'erreurs et je ne sais pas pourquoi!  
j'ai remarqué que le dans fichier.txt , s'est écrit "convertie en DOS" je me dis donc que peut être le format a changé.  
J'ai remarqué aussi qu'un ligne s'importe avec succès et la suivante non, et il y a des \\t qui apparaissent à la place de \t.  
Est ce que vous pourriez m'aider ?  
Merci.

Exemple du log.txt

```auto
20191101 00:00:00.390 MPC CarteAbsente A:3,I:1,P:0,R:0,E:1
20191101 00:00:00.500 MPC GereRecepMPC10 | EOT1 - Msg Reçu :Aa0040000a A:6,I:1,P:0,R:1,E:1

```

Exemple du log.conf

```auto
grok {
	match => ["message","(?<timestamp>%{YEAR:YYYY}%{MONTHNUM:MM}%{MONTHDAY:dd}%{SPACE}%{HOUR}:%{MINUTE}:%{SECOND}.[0-9]{3})%{SPACE}%{GREEDYDATA:data}"]
	}
	date {
		match => ["timestamp", "YYYYMMdd HH:mm:ss.SSS", "YYYYMMdd HH:mm:ss.SSS"]
		target => "@timestamp"
		timezone => "GMT"
		locale => "fr"
	}

```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 20, 2020, 1:39am UTC](https://discuss.elastic.co/t/erreur-dimportation-avec-logstash-sous-linux/233427/2 "2020-05-20T01:39:58Z")

</div>

> [@Katia](#):
>
> Bonjour,  
> J'importe mes logs avec logstash vers elasticserach, ça marche bien en local ( sous windows).  
> Mais pas sous linux, j'ai pleins d'erreurs et je ne sais pas pourquoi!  
> j'ai remarqué que le dans fichier.txt , s'est écrit "convertie en DOS" je me dis donc que peut être le format a changé.  
> J'ai remarqué aussi qu'un ligne s'importe avec succès et la suivante non, et il y a des \t qui apparaissent à la place de \t.  
> Est ce que vous pourriez m'aider ?  
> Merci.

If you want to ask a question in French then you may get a better answer in [another forum](https://discuss.elastic.co/c/in-your-native-tongue/discussions-en-francais/16). I did a Google translate and did not really understand the question about tab conversion. A native speaker may well understand better.

---

<div class="post-metadata">

### Author: ![Katia](https://avatars.discourse-cdn.com/v4/letter/k/b5e925/32.png) [@Katia](https://discuss.elastic.co/u/Katia)
#### Post date: [May 20, 2020, 7:24am UTC](https://discuss.elastic.co/t/erreur-dimportation-avec-logstash-sous-linux/233427/3 "2020-05-20T07:24:32Z")

</div>

> [@Badger](#):
>
> Bonjour,  
> J'importe mes logs avec logstash vers elasticserach, ça marche bien en local ( sous windows).  
> Mais pas sous linux, j'ai pleins d'erreurs et je ne sais pas pourquoi!  
> j'ai remarqué que le dans fichier.txt , s'est écrit "convertie en DOS" je me dis donc que peut être le format a changé.  
> J'ai remarqué aussi qu'un ligne s'importe avec succès et la suivante non, et il y a des \t qui apparaissent à la place de \t.  
> Est ce que vous pourriez m'aider ?  
> Merci.

Can I delete this post and ask another the same question in english ?

---

<div class="post-metadata">

### Author: ![mcardlesam](https://avatars.discourse-cdn.com/v4/letter/m/ee7513/32.png) [@mcardlesam](https://discuss.elastic.co/u/mcardlesam)
#### Post date: [May 20, 2020, 7:32am UTC](https://discuss.elastic.co/t/erreur-dimportation-avec-logstash-sous-linux/233427/4 "2020-05-20T07:32:24Z")

</div>

> [@Katia](#):
>
> Bonjour,  
> J'importe mes logs avec logstash vers elasticserach, ça marche bien en local ( sous windows).  
> Mais pas sous linux, j'ai pleins d'erreurs et je ne sais pas pourquoi!  
> j'ai remarqué que le dans fichier.txt , s'est écrit "convertie en DOS" je me dis donc que peut être le format a changé.  
> J'ai remarqué aussi qu'un ligne s'importe avec succès et la suivante non, et il y a des \t qui apparaissent à la place de \t.  
> Est ce que vous pourriez m'aider ?  
> Merci.

> Hello,  
> I import my logs with logstash to elasticserach, it works well locally (under windows).  
> But not under Linux, I have lots of errors and I don't know why!  
> I noticed that the in file.txt, was written "converted to DOS" so I tell myself that maybe the format has changed.  
> I also noticed that one line imports successfully and the next does not, and there are \ t that appear in place of \ t.  
> Can you help me?  
> Thank you.

Can you post the errors you are getting? If you output the log file in a terminal in linux (`cat file.txt`) does it correctly print multiple lines?

---

<div class="post-metadata">

### Author: ![Katia](https://avatars.discourse-cdn.com/v4/letter/k/b5e925/32.png) [@Katia](https://discuss.elastic.co/u/Katia)
#### Post date: [May 20, 2020, 8:22am UTC](https://discuss.elastic.co/t/erreur-dimportation-avec-logstash-sous-linux/233427/5 "2020-05-20T08:22:20Z")

</div>

There is a difference in the message field

```auto
{
          "data" => "MPC CarteAbsente A:3,I:1,P:0,R:0,E:1",
     "timestamp" => "20191101\t04:24:38.484",
       "message" => "20191101\t04:24:38.484\tMPC CarteAbsente A:3,I:1,P:0,R:0,E:1",
    "@timestamp" => 2019-11-01T04:24:38.484Z,
           "seq" => 58175,
          "host" => "ns510529"
}
{
       "message" => "20191101\\t04:24:38.593\\tMPC GereRecepMPC10 | EOT1 - Msg Re\\xE7u :\\u0002Aa0040000\\u0010\\u0003\\a A:6,I:1,P:0,R:1,E:1",
    "@timestamp" => 2020-05-20T08:18:53.952Z,
          "tags" => [
        [0] "_grokparsefailure"
    ],
           "seq" => 58176,
          "host" => "ns510529"
}

```

---

<div class="post-metadata">

### Author: ![mcardlesam](https://avatars.discourse-cdn.com/v4/letter/m/ee7513/32.png) [@mcardlesam](https://discuss.elastic.co/u/mcardlesam)
#### Post date: [May 20, 2020, 9:36am UTC](https://discuss.elastic.co/t/erreur-dimportation-avec-logstash-sous-linux/233427/6 "2020-05-20T09:36:55Z")

</div>

I am not 100% sure but i think the issue is with parsing the unicode characters in that second line

I think the `\\t` in the message is misleading as on failure grok will put the entire line in the message and is probably re escaping the `\t` in the log line

I think you need to specify the charset as UTF-8 - but Im not sure how to do that. Something like  
[https://www.elastic.co/guide/en/logstash/current/plugins-codecs-plain.html](https://www.elastic.co/guide/en/logstash/current/plugins-codecs-plain.html)

---

<div class="post-metadata">

### Author: ![Katia](https://avatars.discourse-cdn.com/v4/letter/k/b5e925/32.png) [@Katia](https://discuss.elastic.co/u/Katia)
#### Post date: [May 21, 2020, 10:01am UTC](https://discuss.elastic.co/t/erreur-dimportation-avec-logstash-sous-linux/233427/7 "2020-05-21T10:01:34Z")

</div>

I found this :

```auto
codec => plain {
      charset => "name of charset here"
    }

```

but having already used

```auto
codec => multiline {
		}

```

I don't know how to use both?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 21, 2020, 2:52pm UTC](https://discuss.elastic.co/t/erreur-dimportation-avec-logstash-sous-linux/233427/8 "2020-05-21T14:52:23Z")

</div>

> [@Katia](#):
>
> I don't know how to use both?

You cannot use both, pick one or the other.

---

<div class="post-metadata">

### Author: ![Katia](https://avatars.discourse-cdn.com/v4/letter/k/b5e925/32.png) [@Katia](https://discuss.elastic.co/u/Katia)
#### Post date: [May 21, 2020, 3:16pm UTC](https://discuss.elastic.co/t/erreur-dimportation-avec-logstash-sous-linux/233427/9 "2020-05-21T15:16:38Z")

</div>

> [@Katia](#):
>
> codec =\> multiline { }

I tried

```auto
codec => multiline {
                 charset => "UTF-8",

		}

```

But in this case, logstash don't run !!!

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 21, 2020, 3:20pm UTC](https://discuss.elastic.co/t/erreur-dimportation-avec-logstash-sous-linux/233427/10 "2020-05-21T15:20:26Z")

</div>

> [@Katia](#):
>
> codec =\> multiline { charset =\> "UTF-8", }

Remove the trailing comma.

---

<div class="post-metadata">

### Author: ![Katia](https://avatars.discourse-cdn.com/v4/letter/k/b5e925/32.png) [@Katia](https://discuss.elastic.co/u/Katia)
#### Post date: [May 27, 2020, 7:53pm UTC](https://discuss.elastic.co/t/erreur-dimportation-avec-logstash-sous-linux/233427/11 "2020-05-27T19:53:58Z")

</div>

i still have error message and logstash is wont run!

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 27, 2020, 8:11pm UTC](https://discuss.elastic.co/t/erreur-dimportation-avec-logstash-sous-linux/233427/12 "2020-05-27T20:11:33Z")

</div>

What error message?

---

<div class="post-metadata">

### Author: ![Katia](https://avatars.discourse-cdn.com/v4/letter/k/b5e925/32.png) [@Katia](https://discuss.elastic.co/u/Katia)
#### Post date: [May 27, 2020, 8:28pm UTC](https://discuss.elastic.co/t/erreur-dimportation-avec-logstash-sous-linux/233427/13 "2020-05-27T20:28:07Z")

</div>

I can't understand it!

```auto
warning: thread "Converge PipelineAction::Create<main>" terminated with exception (report_on_exception is true):
LogStash::Error: Don't know how to handle `Java::JavaLang::IllegalStateException` for `PipelineAction::Create<main>`
          create at org/logstash/execution/ConvergeResultExt.java:129
             add at org/logstash/execution/ConvergeResultExt.java:57
  converge_state at /usr/share/logstash/logstash-core/lib/logstash/agent.rb:355
[ERROR] 2020-05-27 22:24:51.414 [Agent thread] agent - An exception happened when converging configuration {:exception=>LogStash::Error, :message=>"Don't know how to handle `Java::JavaLang::IllegalStateException` for `PipelineAction::Create<main>`", :backtrace=>["org/logstash/execution/ConvergeResultExt.java:129:in `create'", "org/logstash/execution/ConvergeResultExt.java:57:in `add'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:355:in `block in converge_state'"]}
[FATAL] 2020-05-27 22:24:51.437 [LogStash::Runner] runner - An unexpected error occurred! {:error=>#<LogStash::Error: Don't know how to handle `Java::JavaLang::IllegalStateException` for `PipelineAction::Create<main>`>, :backtrace=>["org/logstash/execution/ConvergeResultExt.java:129:in `create'", "org/logstash/execution/ConvergeResultExt.java:57:in `add'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:355:in `block in converge_state'"]}
[ERROR] 2020-05-27 22:24:51.452 [LogStash::Runner] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

```

and my code is

```auto
input{
        file {
        path => ["/home/katia/FichiersLog/SAMIA/h8.txt"]
                start_position => beginning
                codec => multiline {
                        charset => 'ASCII_8BIT'
                        pattern => "^%{YEAR}%{MONTHNUM2}%{MONTHDAY}"
                        negate => true
                        what => previous
                }
    }
}

```

```auto

```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 27, 2020, 9:04pm UTC](https://discuss.elastic.co/t/erreur-dimportation-avec-logstash-sous-linux/233427/14 "2020-05-27T21:04:06Z")

</div>

Can you try adding

```
--java-execution false

```

to the command line and see if you get a better error message?

---

<div class="post-metadata">

### Author: ![Katia](https://avatars.discourse-cdn.com/v4/letter/k/b5e925/32.png) [@Katia](https://discuss.elastic.co/u/Katia)
#### Post date: [May 27, 2020, 11:27pm UTC](https://discuss.elastic.co/t/erreur-dimportation-avec-logstash-sous-linux/233427/15 "2020-05-27T23:27:54Z")

</div>

```auto
[INFO] 2020-05-28 01:24:11.830 [LogStash::Runner] runner - Starting Logstash {"logstash.version"=>"7.7.0"}
[ERROR] 2020-05-28 01:24:13.364 [Converge PipelineAction::Create<main>] agent - Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [\\t\\r\\n], \"#\", \"=>\" at line 6, column 12 (byte 119) after input{\n\tfile {\n path => [\"/home/katia/g2.txt\"]\n\t\tstart_position => beginning\n\t\tcodec => multiline {\n\t\t\tcharset ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:58:in `compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:66:in `compile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:28:in `block in compile_sources'", "org/jruby/RubyArray.java:2577:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:27:in `compile_sources'", "org/logstash/execution/AbstractPipelineExt.java:181:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:40:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:112:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:52:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:342:in `block in converge_state'"]}
[INFO] 2020-05-28 01:24:13.678 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}
[INFO] 2020-05-28 01:24:18.555 [LogStash::Runner] runner - Logstash shut down.

```

---

<div class="post-metadata">

### Author: ![Katia](https://avatars.discourse-cdn.com/v4/letter/k/b5e925/32.png) [@Katia](https://discuss.elastic.co/u/Katia)
#### Post date: [May 27, 2020, 11:39pm UTC](https://discuss.elastic.co/t/erreur-dimportation-avec-logstash-sous-linux/233427/16 "2020-05-27T23:39:27Z")

</div>

Now it's officially an encryption problem.  
Problem lines contain characters like: ç ,ô, ' ...  
and, in my CentOS, those characters appear like this "?"

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 27, 2020, 11:48pm UTC](https://discuss.elastic.co/t/erreur-dimportation-avec-logstash-sous-linux/233427/17 "2020-05-27T23:48:36Z")

</div>

> [@Katia](#):
>
> `:message=>"Expected one of [\\t\\r\\n], \"#\", \"=>\" at line 6, column 12 (byte 119) after input{\n\tfile {\n path => [\"/home/katia/g2.txt\"]\n\t\tstart_position => beginning\n\t\tcodec => multiline {\n\t\t\tcharset "`

What exactly comes after charset in your multiline codec configuration? An error could be caused by it being = rather than =\>

---

<div class="post-metadata">

### Author: ![Katia](https://avatars.discourse-cdn.com/v4/letter/k/b5e925/32.png) [@Katia](https://discuss.elastic.co/u/Katia)
#### Post date: [May 28, 2020, 12:12am UTC](https://discuss.elastic.co/t/erreur-dimportation-avec-logstash-sous-linux/233427/18 "2020-05-28T00:12:52Z")

</div>

Ops, that's right. I put = and not =\>  
Now logstash is running, i still have some warning like :

```auto
[WARN] 2020-05-28 02:05:43.777 [[main]<file] multiline - Received an event that has a different character encoding than you configured. {:text=>"20191101\\t02:42:10.421\\tMPC GereRecepMPC10 | EOT1 - Msg Re\\xE7u :\\u0002Aa0040000\\u0010\\u0003\\a A:6,I:1,P:0,R:1,E:1\\r", :expected_charset=>"UTF-8"}

```

We can see that "Msg Reçu" becomes Msg Re\xE7u...  
I think I need to put something other than UTF-8

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 28, 2020, 12:32am UTC](https://discuss.elastic.co/t/erreur-dimportation-avec-logstash-sous-linux/233427/19 "2020-05-28T00:32:42Z")

</div>

> [@Katia](#):
>
> I think I need to put something other than UTF-8

I agree. I am not an expert in character encodings so I cannot tell you what else to use.

---

<div class="post-metadata">

### Author: ![Katia](https://avatars.discourse-cdn.com/v4/letter/k/b5e925/32.png) [@Katia](https://discuss.elastic.co/u/Katia)
#### Post date: [May 28, 2020, 12:37am UTC](https://discuss.elastic.co/t/erreur-dimportation-avec-logstash-sous-linux/233427/20 "2020-05-28T00:37:04Z")

</div>

Problem solved.  
my final code is:

```auto
input{
        file {
        path => ["/home/katia/FichiersLog/SAMIA/gest1.txt"]
                start_position => beginning
                codec => multiline {
                        charset => "BINARY"
                        pattern => "^%{YEAR}%{MONTHNUM2}%{MONTHDAY}"
                        negate => true
                        what => previous
                }
    }
}

```

thank you very much for your time.

[Next page](https://discuss.elastic.co/t/erreur-dimportation-avec-logstash-sous-linux/233427.md?page=2)
