# Atlassian access logs Index not getting created or data not sent / visible in Opensearch

**URL:** <https://discuss.elastic.co/t/atlassian-access-logs-index-not-getting-created-or-data-not-sent-visible-in-opensearch/341742>\
**Category:** Logstash\
**Created:** [August 26, 2023, 11:17am UTC](https://discuss.elastic.co/t/atlassian-access-logs-index-not-getting-created-or-data-not-sent-visible-in-opensearch/341742 "2023-08-26T11:17:40Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![danmed](https://avatars.discourse-cdn.com/v4/letter/d/aeb1de/32.png) [@danmed](https://discuss.elastic.co/u/danmed)\
**Post date:** [August 26, 2023, 11:17am UTC](https://discuss.elastic.co/t/atlassian-access-logs-index-not-getting-created-or-data-not-sent-visible-in-opensearch/341742/1 "2023-08-26T11:17:40Z")

</div>

I'm trying to use Logstash to send Atlassian access logs to opensearch.

I'm absolutely new to the topic but can successfully send other logs and view them.

It's the jira access logs that I cannot make work.  
Having tried a lot in vain, I'm hoping to get an answer in the community. Please help...

My conf file for the jira access log is below:

```auto
input {
  file {
    path => "/path/to/jira/application/logs/access_log.%{+YYYY-MM-dd}"
    codec => multiline {
      pattern => "^\s"
      what => "previous"
      }
    }
}

filter {
    grok {
            match => { "message" => "^%{IP:ip}\s+%{WORD:userid}\s+%{USER:user}\s+\[%{HTTPDATE:timestamp}\]\s+%{GREEDYDATA:message}" }
    }
    date {
        match => ["timestamp" , "dd/MMM/yyyy:HH:mm:ss Z"]
    }
}

output {
  opensearch {
    hosts => ["https://nn.nnn.nn.nn:9200"]
    ssl_certificate_verification => false
    index => "jira-access-log-%{+YYYY.MM.dd}"
    user => "logstash"
    password => "somepassword"
  }
}

```

The example log lines are below:

```auto
NNN.NNN.NNN.NN 123x456x789 - [26/Aug/2023:12:53:21 +0200] "GET /rest/capabilities/navigation?lang=de-DE HTTP/1.0" 200 1117 8 "-" "Confluence-7.19.12 (18035)" "-"
NNN.NNN.NNN.NN 901x2345x67890 some.string.text [26/Aug/2023:12:55:13 +0200] "PUT /rest/api/latest/issue/DUMMYIS-1234- HTTP/1.0" 400 123 13 "-" "Atlassian HttpClient 0.23.0 / Atlassian JIRA Rest Java Client-4.0.3-sc (0) / Default" "username"

```

I tried the filter plugin to use grok (and it itself seems to work because I have tried it for at least one log line example shown above. I added the date but I'm not sure if it's right/needed...

I also tried to use dissect but I am not sure if it worked...

```auto
filter {
  dissect {
    mapping => { "message" => "%{ip} %{id} %{user} [%{[@metadata][timestamp]} %{timezone}] %{message}" }
  }
  date {
    match => ["[@metadata][timestamp]", "dd-MMM-yyyy:HH:mm:ss" ] #is this correct?
    target => "@timestamp"
  }
}

```

Logstash starts without errors and the pipeline is successfully created.

However I'm not sure if I did it right because I cannot create an index pattern in opensearch..  
I get: "The index pattern you've entered doesn't match any indices."

So I'm not sure if the index is even getting created / data being sent. And whether I need to do anything else in opensearch itself...?

I'd really appreciate it if someone could please help me figure this out? Thanks so much,  
Dan

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 26, 2023, 11:17am UTC](https://discuss.elastic.co/t/atlassian-access-logs-index-not-getting-created-or-data-not-sent-visible-in-opensearch/341742/2 "2023-08-26T11:17:40Z")

</div>

OpenSearch/OpenDistro are AWS run products and differ from the original Elasticsearch and Kibana products that Elastic builds and maintains. You may need to contact them directly for further assistance.

(This is an automated response from your friendly Elastic bot. Please report this post if you have any suggestions or concerns :elasticheart: )

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [August 26, 2023, 12:15pm UTC](https://discuss.elastic.co/t/atlassian-access-logs-index-not-getting-created-or-data-not-sent-visible-in-opensearch/341742/3 "2023-08-26T12:15:43Z")

</div>

Welcome to the community.

This is a single line log, no need for the multiline codec.  
Also you can improve grok with more fields if you like.

```auto
input {
  file {
   path => "/path/to/jira/application/logs/access_log.*"
   start_position => beginning
   #sincedb_path => "/dev/null" # if you like add comment to track record about lines in the default sincedb or /dev/null to keep in memory(useful for the testing)
   }
}

```

---

<div class="post-metadata">

**Author:** ![danmed](https://avatars.discourse-cdn.com/v4/letter/d/aeb1de/32.png) [@danmed](https://discuss.elastic.co/u/danmed)\
**Post date:** [August 26, 2023, 12:26pm UTC](https://discuss.elastic.co/t/atlassian-access-logs-index-not-getting-created-or-data-not-sent-visible-in-opensearch/341742/4 "2023-08-26T12:26:25Z")

</div>

Thanks Rios, I'll remove the multiline codec.

Is the current grok somehow preventing the index creation?  
How would I know the index is being being created?  
I don't see it in opensearch's Dev Tools using "GET /\_cat/indices"  
Thanks

---

<div class="post-metadata">

**Author:** ![danmed](https://avatars.discourse-cdn.com/v4/letter/d/aeb1de/32.png) [@danmed](https://discuss.elastic.co/u/danmed)\
**Post date:** [August 26, 2023, 1:20pm UTC](https://discuss.elastic.co/t/atlassian-access-logs-index-not-getting-created-or-data-not-sent-visible-in-opensearch/341742/5 "2023-08-26T13:20:40Z")

</div>

And also if I put nothing in the filter:

filter {  
}

I still don't see the index in opensearch, so I can't create any index pattern for it...

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [August 26, 2023, 1:27pm UTC](https://discuss.elastic.co/t/atlassian-access-logs-index-not-getting-created-or-data-not-sent-visible-in-opensearch/341742/6 "2023-08-26T13:27:12Z")

</div>

Are you sure that logstash is reading the log file correctly?

You need to troubleshoot this first, remove the opensearch output and add a file output or stdout output to see at least if logstash is reading and processing the logs.

Also, how are you running Logstash? As a service? Can you start it again to get fresh logs and share the logs here?

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [August 26, 2023, 1:28pm UTC](https://discuss.elastic.co/t/atlassian-access-logs-index-not-getting-created-or-data-not-sent-visible-in-opensearch/341742/7 "2023-08-26T13:28:18Z")

</div>

If you not set sincedb\_path, LS will keep record about files which are read.  
Try with for test:

```auto
input {
  file {
   path => "/path/to/jira/application/logs/access_log.*"
   start_position => beginning
   sincedb_path => "/dev/null" 
   }
}

```

---

<div class="post-metadata">

**Author:** ![danmed](https://avatars.discourse-cdn.com/v4/letter/d/aeb1de/32.png) [@danmed](https://discuss.elastic.co/u/danmed)\
**Post date:** [August 26, 2023, 4:40pm UTC](https://discuss.elastic.co/t/atlassian-access-logs-index-not-getting-created-or-data-not-sent-visible-in-opensearch/341742/8 "2023-08-26T16:40:00Z")

</div>

Hi leandrojmp, I am running LS as a systemd service. So, yes it would be good to see the output going into a file.

However would I be able to redirect output to a file on the same server on which LS is running?

I ask because I can't ssh login to the target opensearch server for now.

So, would the snip below work?

```auto
output {
  stdout { 
     codec => "json_lines" 
     path => "/tmp/checklogstash.tmp"
   }
}

```

Also which logs do you want me to share? The actual access logs that LS will work on or the logstash-plain.log ?

---

<div class="post-metadata">

**Author:** ![danmed](https://avatars.discourse-cdn.com/v4/letter/d/aeb1de/32.png) [@danmed](https://discuss.elastic.co/u/danmed)\
**Post date:** [August 26, 2023, 4:42pm UTC](https://discuss.elastic.co/t/atlassian-access-logs-index-not-getting-created-or-data-not-sent-visible-in-opensearch/341742/10 "2023-08-26T16:42:44Z")

</div>

Thanks for your suggestion Rios, i will try it out.

---

<div class="post-metadata">

**Author:** ![danmed](https://avatars.discourse-cdn.com/v4/letter/d/aeb1de/32.png) [@danmed](https://discuss.elastic.co/u/danmed)\
**Post date:** [August 26, 2023, 5:22pm UTC](https://discuss.elastic.co/t/atlassian-access-logs-index-not-getting-created-or-data-not-sent-visible-in-opensearch/341742/11 "2023-08-26T17:22:07Z")

</div>

ok so yes the output is to a file on the same server. I just tried it with some test data on a test server... I will check the access log output if it works soon and goes into a file.

---

<div class="post-metadata">

**Author:** ![danmed](https://avatars.discourse-cdn.com/v4/letter/d/aeb1de/32.png) [@danmed](https://discuss.elastic.co/u/danmed)\
**Post date:** [August 27, 2023, 12:59pm UTC](https://discuss.elastic.co/t/atlassian-access-logs-index-not-getting-created-or-data-not-sent-visible-in-opensearch/341742/12 "2023-08-27T12:59:50Z")

</div>

Hi leandrojmp,  
When I use a file output to see at least if LS is reading and processing the logs, the file does **not** get created on my production server. It does however get created in my test server.  
I have rw permissions, there is space on the disk.  
My full config is here including the input, filter and output:

```auto
input {
  file {
    path => "/path/to/jira/acces_log.%{+YYYY-MM-dd}"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}

filter {
    grok {
            match => { "message" => "^%{IP:ip}\s+%{WORD:userid}\s+%{USER:user}|-\s+\[%{HTTPDATE:timestamp}\]\s+%{GREEDYDATA:message}" }
    }
    date {
        match => ["timestamp" , "dd/MMM/yyyy:HH:mm:ss Z"]
    }
}

output {
                file {
                path => "/path/to/logstash.out"
                }
}

```

If I can see the output file and it's content I would know if the input and filter is working before checking why an index is not being created....  
Please help..  
Thanks

---

<div class="post-metadata">

**Author:** ![danmed](https://avatars.discourse-cdn.com/v4/letter/d/aeb1de/32.png) [@danmed](https://discuss.elastic.co/u/danmed)\
**Post date:** [August 27, 2023, 1:08pm UTC](https://discuss.elastic.co/t/atlassian-access-logs-index-not-getting-created-or-data-not-sent-visible-in-opensearch/341742/13 "2023-08-27T13:08:17Z")

</div>

Also if I only have an input and output ( with file ) i.e without any filter section, I still don't see an output file...

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [August 27, 2023, 2:40pm UTC](https://discuss.elastic.co/t/atlassian-access-logs-index-not-getting-created-or-data-not-sent-visible-in-opensearch/341742/14 "2023-08-27T14:40:49Z")

</div>

> [@leandrojmp](#):
>
> Can you start it again to get fresh logs and share the logs here?

You didn't share any logs, please restart logstash, get the logs and share then, this will show if there is any errors or warnings.

> [@danmed](#):
>
> `path => "/path/to/jira/acces_log.%{+YYYY-MM-dd}"`

What is this path? When you run Logstash as a service it will run as the `logstash` user, so you need to make sure that the logstash user can in fact read those logs, check the path permissions.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 27, 2023, 3:47pm UTC](https://discuss.elastic.co/t/atlassian-access-logs-index-not-getting-created-or-data-not-sent-visible-in-opensearch/341742/15 "2023-08-27T15:47:11Z")

</div>

> [@danmed](#):
>
> `/path/to/jira/acces_log.%{+YYYY-MM-dd}"`

The file input does not sprintf the path option, so I don't think this does what you want. I cannot figure out what it does do. It doesn'i tail `/path/to/jira/acces_log.%{+YYYY-MM-dd}`, nor does it tail `/path/to/jira/acces_log.2023-08-27`. Even trace level logging does not make it clear what it is doing.

---

<div class="post-metadata">

**Author:** ![danmed](https://avatars.discourse-cdn.com/v4/letter/d/aeb1de/32.png) [@danmed](https://discuss.elastic.co/u/danmed)\
**Post date:** [August 27, 2023, 4:29pm UTC](https://discuss.elastic.co/t/atlassian-access-logs-index-not-getting-created-or-data-not-sent-visible-in-opensearch/341742/16 "2023-08-27T16:29:46Z")

</div>

Hi Badger, thanks for replying.

My input file is and path is below:

The input describes which log I want to process and I have listed it below too.

These are essentially jira access log files that get rotated daily.

```auto
input {
  file {
    path => "/path/to/the/access_log.%{+YYYY-MM-dd}"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}
ls -l /path/to/the/access_log.2023-08-27
-rw-r----- 1 <user> <group> 1593933 Aug 27 18:19 /path/to/the/access_log.2023-08-27

```

If the file cannot be read or opened it would make sense why there's no output, but I doubt it unless I'm wrong.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [August 27, 2023, 4:50pm UTC](https://discuss.elastic.co/t/atlassian-access-logs-index-not-getting-created-or-data-not-sent-visible-in-opensearch/341742/17 "2023-08-27T16:50:10Z")

</div>

Just noticed it now after Badger comment.

This `path => "/path/to/the/access_log.%{+YYYY-MM-dd}"` does not work, you need to use a glob.

Something like this\> `path => "/path/to/the/access_log.*`, using `%{+YYYY-MM-dd}`, will not work.

Try this:

```auto
input {
  file {
    path => "/path/to/the/access_log.*"
    start_position => "beginning"
    sincedb_path => "/dev/null" # only use this when testing, when it works remove this line
  }
}

```

> [@danmed](#):
>
> ```auto
> ls -l /path/to/the/access_log.2023-08-27
> -rw-r----- 1 <user> <group> 1593933 Aug 27 18:19 /path/to/the/access_log.2023-08-27
> 
> ```

Does the `logstash` user have permission in this path and this file? If the `logstash` user does not have permission, logstash won't be able to read the files, you are redacting both the path and the user/group, so it is not possible to know. You need to check the permissions.

Also, you didn't share any logs, so it is not possible to know if there is any issue while logstash is starting or not.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 27, 2023, 4:51pm UTC](https://discuss.elastic.co/t/atlassian-access-logs-index-not-getting-created-or-data-not-sent-visible-in-opensearch/341742/18 "2023-08-27T16:51:57Z")

</div>

> [@danmed](#):
>
> `path => "/path/to/the/access_log.%{+YYYY-MM-dd}"`

The file input does not expand the date reference to the current date. Even if it did then it would be a one time event and logstash would not re-expand it each day.

The standard approach for this would be to use `path => "/path/to/the/access_log.*` and let the file input track old files and rollovers in the sincedb.

---

<div class="post-metadata">

**Author:** ![danmed](https://avatars.discourse-cdn.com/v4/letter/d/aeb1de/32.png) [@danmed](https://discuss.elastic.co/u/danmed)\
**Post date:** [August 27, 2023, 5:14pm UTC](https://discuss.elastic.co/t/atlassian-access-logs-index-not-getting-created-or-data-not-sent-visible-in-opensearch/341742/19 "2023-08-27T17:14:41Z")

</div>

Hi leandrojmp and Badger,

Something worked!

I changed the file input path as you both suggested, to

```auto
input {
  file {
    path => "/path/to/the/access_log.*"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}

```

But there are a couple of problems now:  
**(1)**  
I have strange message in the logstash file like:

```auto

[2023-08-27T19:01:15,622][WARN][logstash.codecs.plain][jira-access-pipeline][15ed61b624792bc0909a82bf7de16475dec0fbb231fbff95fef42ce52bfa2802] Received an event that has a different character encoding than you configured. {:text=>"\\xDB\\r\\xF6\\xF0\\xC1\\t\\u0000\\u001C\\x9A<\\xB1w[T\\u0010x#FRT\\xB0L\\xF0ͦ+\\xC8\\xE9\\xB3\\xF9z\\xF1...etc.etc.etc 
 
....etc..\\u0005^\\xB7D\\x82\\xFD\\xA5W\\xE4\\u0001\\u0010\\u0000\\u0017\\xE3۩\\xE6\\x80\\xF2z|\\u0013\\xE9EUDEUQUDQUDUEUEUF(\\x8A\\xAB\\u0011DEE\\u0015EE\\u0015UU\\u0014UTQ\\u0015\\u0014UEUUUEUTQb\\xACQUUUE\\u0011UQEEU\\u0011PUQX\\xAA*\\xAA1QAUUUTEX\\xAA\\x8A\\x88\\xA8\\xAA1c\\u0010b\\xAA\\xAA\\xA8\\xA0\\xA8\\x8A\\b\\xAA**\\xAA\\xA8\\x8A\\xACb\\xA8\\xA8\\xC4ETU\\x8A\\xAD8\\u0000", :expected_charset=>"UTF-8"}
[2023-08-27T19:01:15,953][WARN][logstash.codecs.plain][jira-access-pipeline][15ed61b624792bc0909a82bf7de16475dec0fbb231fbff95fef42ce52bfa2802] Received an event that has a different character encoding than you configured. {:text=>"\\xCE\\xF6\\x86Q\\x8....etc...xFA\\u0019\\xBFBG\\a\\xA4\\x82\\x90R", :expected_charset=>"UTF-8"}

```

**(2)**  
I can see the index being created in opensearch - or atleast I when I go to create index pattern, I can see the correct index name, But after I choose select a primary time field for use with the global time filter as @timestamp, I get an error during creating the index pattern which says:

```auto
Error

No Reason Phrase

```

So I can't create the index. So near 🙂 yet...  
please help with this..  
Thank you.

---

<div class="post-metadata">

**Author:** ![danmed](https://avatars.discourse-cdn.com/v4/letter/d/aeb1de/32.png) [@danmed](https://discuss.elastic.co/u/danmed)\
**Post date:** [August 27, 2023, 5:28pm UTC](https://discuss.elastic.co/t/atlassian-access-logs-index-not-getting-created-or-data-not-sent-visible-in-opensearch/341742/20 "2023-08-27T17:28:17Z")

</div>

Also, leandrojmp, I will removed the sincedb line like you suggested...

I will try a new grok or dissect filter because I am getting a grokparsefailure like:

`tags"=>["_grokparsefailure"], "message"=>"B?\\xAD*7\\x9D$n\\xB4\\xAB\\xDFۥ/D\\xED\\x90\\xE5a\\u0000I\\xE5\\b\\x91\\u0001]\\xDD\\u0010\\xA4\\xED-f\\u0015\\xA2\\x9E\\xBBߓf֮\\xD983!\\xDD Dw......etc....etc.....99\\xC2GuT\\xF1\\u0017", "host"=>{"name"=>"myhostname.com"}}], :error=>{"type"=>"cluster_block_exception", "reason"=>"index [jira-access-log-2023.08.27] blocked by: [TOO_MANY_REQUESTS/12/disk usage exceeded flood-stage watermark, index has read-only-allow-delete block];"}}`

---

<div class="post-metadata">

**Author:** ![danmed](https://avatars.discourse-cdn.com/v4/letter/d/aeb1de/32.png) [@danmed](https://discuss.elastic.co/u/danmed)\
**Post date:** [August 27, 2023, 5:32pm UTC](https://discuss.elastic.co/t/atlassian-access-logs-index-not-getting-created-or-data-not-sent-visible-in-opensearch/341742/21 "2023-08-27T17:32:11Z")

</div>

And, I noticed it's also 'processing' the zipped rotated files because I used '\*' in the path.  
Does it unzip these before processing?

Since my logs are rotated, I have

access\_log.2023-08-27 and I also have  
access\_log.2023-08-26.bz2  
access\_log.2023-08-25.bz2 etc

If it's reading compressed input, it might not grok...  
Waiting for your thoughts, Thanks again.

[Next page](https://discuss.elastic.co/t/atlassian-access-logs-index-not-getting-created-or-data-not-sent-visible-in-opensearch/341742.md?page=2)