### Author:  [@onlinekap](https://discuss.elastic.co/u/onlinekap)
#### Post date: [August 16, 2021, 6:39am UTC](https://discuss.elastic.co/t/logstash-data-unload/281506/1 "2021-08-16T06:39:34Z")
Hello, I am newbie here.. I have a bunch of ELK indexes that returns below output when querying via POSTMAN GET request.
\
{
"took": 5,
"timed\_out": false,
"\_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max\_score": 9.650149,
"hits": [
{
"\_index": "inx-mon-2020.11.45.311-v1",
"\_type": "\_doc",
"\_id": "8wxU8uclg3bUmpdjyAIw",
"\_score": 9.650149,
"\_source": {
"createDate": "2020-11-06T12:00:00.000+1100",
"createdBy": "user1",
"aZoneId": "Zone1",
"itemId": "ITEMXYZ",
"mId": "MU",
"pickQty": 28,
"searchBaseType": "uomSearchable",
"searchType": "uomPickedSearchable",
"domain": {
"version": "2020.1.4"
}
}
}
]
}
}
\
Then I want the above Elasticsearch output inserted into a table in sql server
Below my ELK config file.
\
input
{
elasticsearch {
hosts =\> ["localhost:9200"]
index =\> "inx-mon-2020.11.45.311-v1"
user =\> "root"
password =\> "\*\*\*\*"
}
}
output {
jdbc{
driver\_jar\_path =\>"/usr/share/logstash/bin/sqljdbc\_7.4/enu/mssql-jdbc-7.4.1.jre11.jar"
connection\_string =\> "jdbc:sqlserver://localhost:1433;databaseName=mydatabase;user=myuser;password=mypass"
statement =\> ["INSERT into mydatabase.dbo.mytable ([timestamp]
,[elkindex]
,[id]
,[createDate]
,[createdBy]
,[aZoneId]
,[itemId]
,[mId]
,[pickQty]) VALUES(?,?,?,?,?,?,?,?,?)", "@timestamp", "[hits][\_index]", "[hits][\_id]", "createDate", "createdBy", "aZoneId", "itemId", "mId", "pickQty" ]
}
stdout { codec =\> rubydebug }
}
\
All working but I want the "\_index" which has the value "inx-mon-2020.11.45.311-v1" captured in the my insert statement under the column elkindex.
How would do this?
# I get the below debug info.. See there is no reference to the header data in insert statement, such as below three fields.
# "\_index": "inx-mon-2020.11.45.311-v1", "\_type": "\_doc", "\_id": "8wxU8uclg3bUmpdjyAIw",
INSERT into mytable ([timestamp]\n ,[elkindex]\n ,[id]\n ,[createDate]\n ,[createdBy]\n ,[aZoneId]\n ,[itemId]\n ,[uomId]\n ,[pickQty]) VALUES(?,?,?,?,?,?,?,?,?)", :event=\>"{"aZoneId":"Zone1","searchBaseType":"uomSearchable","searchType":"uomPickedSearchable","itemId":"ITEMXYZ","@version":"1","mId":"MU","createdBy":"user1","domain":{"version":"2020.1.4"},"pickQty":28,"createDate":"2020-11-06T12:00:00.000+1100","@timestamp":"2021-08-15T11:48:51.805Z"}"}
Thank you for your reply.
---
### Author:  [@online](https://discuss.elastic.co/u/online)
#### Post date: [August 16, 2021, 8:00am UTC](https://discuss.elastic.co/t/logstash-data-unload/281506/2 "2021-08-16T08:00:49Z")
Please note, the above config file is for Logstash.
I found a similar issue under the below thread.
> [@Elasticsearch output: LogStash::Error: timestamp field is missing](https://discuss.elastic.co/t/elasticsearch-output-logstash-timestamp-field-is-missing/197888):
>
> Hello, I have an elasticsearch output working without any problem elasticsearch { hosts =\> ["${LOGS\_ELASTICSEARCH\_CLUSTER}:443"] ssl =\> true index =\> "cwl-%{+YYYY.MM.dd}" document\_type =\> "log" } I wanted to redirect some type of logs to a new elasticsearch index but I'm getting this error [2019-09-03T14:39:00,242][FATAL][logstash.runner] An unexpected error occurred! {:error=\>#\, :backtrace=\>["org/logstash/ext/JrubyEventExtLibrary.java:…
I have to now figure out how to create a [@metadata] field and sprintf to at least access the index name. Any pointers would be highly appreciated.
I think I am having the below issue in logstash compared to Postman where it outputs the additional metadata. How would include them in logstash?
> **[@metadata is missing in Logstash | Filebeat Reference \[8.11\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/metadata-missing.html)**
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 17, 2021, 3:55am UTC](https://discuss.elastic.co/t/logstash-data-unload/281506/3 "2021-08-17T03:55:56Z")
You should learn a little bit of [markdown](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet) if you are going to ask for help here. You taking a few minutes to do that saves the people helping you a lot more than a few minutes.
Using multiple logins and replying to yourself is not a good look. It will discourage people from replying, because it looks like spam.
Your links are confusing. Are you talking about creating [@metadata] fields in filebeat or in logstash? sprintf references in logstash are documented [here](https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#sprintf).
---
### Author:  [@online](https://discuss.elastic.co/u/online)
#### Post date: [August 17, 2021, 5:22am UTC](https://discuss.elastic.co/t/logstash-data-unload/281506/4 "2021-08-17T05:22:45Z")
Thanks Badger, Apologies for multiple logins. I thought the original login wouldn't work as the confirmation email was blocked by the company server earlier. Then ended up creating another one. In the end both got working and seems to have logged in with different tabs of the browser with both logins. Going forward I will be using only one. I am slowly getting my head around this.
It's the logstash fields that aren't visible in the output that I'm after. Specifically the \_index, \_id fields as in the JSON output above with the tag "hits". Even though those two fields are visible via the GET request, the logstash seems to be dropping them. Thanks again.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 17, 2021, 2:31pm UTC](https://discuss.elastic.co/t/logstash-data-unload/281506/5 "2021-08-17T14:31:57Z")
Please edit your original post. Select the document returned by elasticsearch and click on \ in the toolbar above the edit pane. Then do the same for the logstash configuration, then do the same for the debug info. Make sure you do not have the regular text selected when you do this.
When you click on \ you will see the format change from
input
{
elasticsearch {
hosts =\> ["localhost:9200"]
index =\> "inx-mon-2020.11.45.311-v1"
to
```auto
input
{
elasticsearch {
hosts => ["localhost:9200"]
index => "inx-mon-2020.11.45.311-v1"
```
which is far easier to read.
---
### Author:  [@online](https://discuss.elastic.co/u/online)
#### Post date: [August 18, 2021, 1:56am UTC](https://discuss.elastic.co/t/logstash-data-unload/281506/6 "2021-08-18T01:56:37Z")
It seems the editing of the original post is disabled. I tried it under my other login too and there is no audit option. Cheers.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 18, 2021, 2:56am UTC](https://discuss.elastic.co/t/logstash-data-unload/281506/7 "2021-08-18T02:56:53Z")
At the bottom of the post, two icons to the left of Reply, there should be a pencil icon that allows you to edit the post.
---
### Author:  [@online](https://discuss.elastic.co/u/online)
#### Post date: [August 18, 2021, 3:06am UTC](https://discuss.elastic.co/t/logstash-data-unload/281506/8 "2021-08-18T03:06:05Z")
See the attached file
The pencil icon is missing in the list. It's the same under if I use my other login(onlinekap) which was used to post the original question. But all the subsequent posting under the login: online has the pencil icon.
---
### Author:  [@online](https://discuss.elastic.co/u/online)
#### Post date: [August 18, 2021, 7:30am UTC](https://discuss.elastic.co/t/logstash-data-unload/281506/9 "2021-08-18T07:30:17Z")
Thank you Badger, The issue is sorted and the answer is too trivial to mention here as its one liner.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 15, 2021, 7:30am UTC](https://discuss.elastic.co/t/logstash-data-unload/281506/10 "2021-09-15T07:30:33Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.