# Unable to write to avro file from logstash

**URL:** https://discuss.elastic.co/t/unable-to-write-to-avro-file-from-logstash/42410
**Category:** Logstash
**Created:** [February 22, 2016, 4:54pm UTC](https://discuss.elastic.co/t/unable-to-write-to-avro-file-from-logstash/42410 "2016-02-22T16:54:36Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![sdaruna](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sdaruna/32/7289_2.png) [@sdaruna](https://discuss.elastic.co/u/sdaruna)
#### Post date: [February 22, 2016, 4:54pm UTC](https://discuss.elastic.co/t/unable-to-write-to-avro-file-from-logstash/42410/1 "2016-02-22T16:54:36Z")

</div>

Hi,

I am trying to write the file using avro codel in logstash.

Below is my simple configuration:

===  
input {  
file {  
path =\> "/Users/srini/Downloads/imp/2016-01-22/\*.json"  
start\_position =\> "beginning"  
}  
}

```
output {
file {
  path => "/Users/srini/Downloads/imp/2016-01-32/"
  codec => avro {schema_uri => "/Users/srini/Sri_Search/av.avsc"}
 }
  stdout { codec => rubydebug }
}

```

===

After this i have copied a json file to the target folder.  
But, nothing is happening.

Below is the json and avsc files.

Json:

`{"username":"miguno","tweet":"Rock: Nerf paper, scissors is fine.","timestamp": 1366150681 }`

Avsc file:

```
{
  "type" : "record",
  "name" : "twitter_schema",
  "namespace" : "com.miguno.avro",
  "fields" : [ {
    "name" : "username",
    "type" : "string",
    "doc" : "Name of the user account on Twitter.com"
  }, {
    "name" : "tweet",
    "type" : "string",
    "doc" : "The content of the user's Twitter message"
  }, {
    "name" : "timestamp",
    "type" : "long",
    "doc" : "Unix epoch time in seconds"
  } ],
  "doc:" : "A basic schema for storing Twitter messages"
}

```

1. Can we use codec with output format.?
2. Why it is unable to take any data.? I have movied a new file to the folder. At least it is not throwing any error.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 22, 2016, 6:05pm UTC](https://discuss.elastic.co/t/unable-to-write-to-avro-file-from-logstash/42410/2 "2016-02-22T18:05:44Z")

</div>

The destination file (the `path` option of your file output) is a directory. I'm pretty sure it needs to be a path to a file.

---

<div class="post-metadata">

### Author: ![sdaruna](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sdaruna/32/7289_2.png) [@sdaruna](https://discuss.elastic.co/u/sdaruna)
#### Post date: [February 22, 2016, 6:09pm UTC](https://discuss.elastic.co/t/unable-to-write-to-avro-file-from-logstash/42410/3 "2016-02-22T18:09:40Z")

</div>

HI @magnusbaeck

I have changed the configuration to

```
input {
    file {
        path => "/Users/srini/Downloads/imp/2016-01-22/*.json"
        start_position => "beginning"
    }
}

output {
file {
  path => "/Users/srini/Downloads/imp/2016-01-31/test-%{+YYYY-MM-dd}.avro"
  codec => avro { schema_uri => "/Users/srini/Sri_Search/av.avsc" }
 }
  stdout { codec => rubydebug }
} 

```

and restarted the logstash and copied new file into folder. But, still no use.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 22, 2016, 6:45pm UTC](https://discuss.elastic.co/t/unable-to-write-to-avro-file-from-logstash/42410/4 "2016-02-22T18:45:22Z")

</div>

Read the Logstash logs. If there's nothing interested, increase logging verbosity with `--verbose`.

---

<div class="post-metadata">

### Author: ![sdaruna](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sdaruna/32/7289_2.png) [@sdaruna](https://discuss.elastic.co/u/sdaruna)
#### Post date: [February 22, 2016, 6:51pm UTC](https://discuss.elastic.co/t/unable-to-write-to-avro-file-from-logstash/42410/5 "2016-02-22T18:51:10Z")

</div>

Hi @magnusbaeck

I have tried that.. I gave --verbose and nothing has been printed. I have also deleted .sincedb\* (though irrelevant, just to see what it can do).

However, if i gave wrong file that does not match the format, it is throwing error. But nothing is being printed/done for the right files.

Did you get a chance to try with the files that i have given.?

Thank You,  
Regards,  
Srini.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 22, 2016, 6:54pm UTC](https://discuss.elastic.co/t/unable-to-write-to-avro-file-from-logstash/42410/6 "2016-02-22T18:54:49Z")

</div>

> Did you get a chance to try with the files that i have given.?

No, I don't have time to do that.

If I were to debug this, I'd make the file input and outputs work okay independently from each other, i.e. first define a stdin input as the only input and pass it data that you know is correct and thus verify that the output works. Perhaps use a stdout output with an avro codec instead of a file output. Increase logging even more with `--debug` as necessary.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 5:10am UTC](https://discuss.elastic.co/t/unable-to-write-to-avro-file-from-logstash/42410/7 "2017-07-06T05:10:18Z")

</div>


