# Logstash conf output to elasticsearch with document\_id set only indexes one record

**URL:** <https://discuss.elastic.co/t/logstash-conf-output-to-elasticsearch-with-document-id-set-only-indexes-one-record/272604>\
**Category:** Logstash\
**Created:** [May 10, 2021, 5:21pm UTC](https://discuss.elastic.co/t/logstash-conf-output-to-elasticsearch-with-document-id-set-only-indexes-one-record/272604 "2021-05-10T17:21:02Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![andcur](https://avatars.discourse-cdn.com/v4/letter/a/2acd7d/32.png) [@andcur](https://discuss.elastic.co/u/andcur)\
**Post date:** [May 10, 2021, 5:21pm UTC](https://discuss.elastic.co/t/logstash-conf-output-to-elasticsearch-with-document-id-set-only-indexes-one-record/272604/1 "2021-05-10T17:21:02Z")

</div>

Hi,

I am using logstash to populate data from SQL server to elastic.

When I specify document\_id (as below) I only get one document in my index. If I do not specify that document\_id field should be taken from PolicyId, I get all the records. Any ideas?

```auto
# Sample Logstash configuration for creating a simple
# Beats -> Logstash -> Elasticsearch pipeline.

input {
  beats {
    port => 5044
  }
}

input {  
	jdbc {
		# SqlServer jdbc connection string to your database, productdb
		# "jdbc:sqlserver://HostName\instanceName;database=DBName;user=UserName;password=Password" 
		jdbc_connection_string => "jdbc:sqlserver://10.130.40.20;database=ApiTeamExposureHub;user=exposure;password=test@123"
		# The user you want to execute your statement as
		jdbc_user => nil
		# The path to your downloaded jdbc driver
		# jdbc_driver_library => "C:/Program Files/sqljdbc_6.0/enu/jre8/sqljdbc42.jar"
		jdbc_driver_library => "C:\Program Files\Microsoft JDBC DRIVER 9.2 for SQL Server\sqljdbc_9.2.1.0_enu\sqljdbc_9.2\enu\mssql-jdbc-9.2.1.jre11.jar"
		# The name of the driver class for SqlServer
		jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
		# Query for testing purpose
		statement => "portfolio.usp_Policy_SelectListForElasticsearch"
	}
}

output {
  elasticsearch {
    hosts => ["https://UKEE-EXP004:9200"]
    index => "policylocalexposurehub"
    user => "elastic"
    password => "p7p8qHk9GdU0v0RLUDM9"
	ssl => true
    cacert => 'config\elasticsearch-ca.pem'
	document_id => "%{PolicyId}"
  }
}

```

[https://UKEE-EXP004:9200/policylocalexposurehub/\_count](https://UKEE-EXP004:9200/policylocalexposurehub/_count)

```auto
{
    "count": 1,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    }
}

```

![image](https://us1.discourse-cdn.com/elastic/original/3X/1/c/1cacd554a413b15e861c83af203b8aed417cfdfb.png)

---

<div class="post-metadata">

**Author:** ![andcur](https://avatars.discourse-cdn.com/v4/letter/a/2acd7d/32.png) [@andcur](https://discuss.elastic.co/u/andcur)\
**Post date:** [May 10, 2021, 5:41pm UTC](https://discuss.elastic.co/t/logstash-conf-output-to-elasticsearch-with-document-id-set-only-indexes-one-record/272604/2 "2021-05-10T17:41:37Z")

</div>

Mappings are all transformed into lower case so the document\_id needs to be specified in lower case.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 7, 2021, 5:42pm UTC](https://discuss.elastic.co/t/logstash-conf-output-to-elasticsearch-with-document-id-set-only-indexes-one-record/272604/3 "2021-06-07T17:42:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.