# Logstash not sends all data to ES
**URL:** https://discuss.elastic.co/t/logstash-not-sends-all-data-to-es/379380
**Category:** Logstash
**Created:** [June 21, 2025, 9:32am UTC](https://discuss.elastic.co/t/logstash-not-sends-all-data-to-es/379380 "2025-06-21T09:32:36Z")
**Posts on this page:** 2
**Page:** 1
### Author:  [@Mercato\_Dev](https://discuss.elastic.co/u/Mercato_Dev)
#### Post date: [June 21, 2025, 9:32am UTC](https://discuss.elastic.co/t/logstash-not-sends-all-data-to-es/379380/1 "2025-06-21T09:32:36Z")
I have 262K item in DB but logstash send only 228K to ES
this is the logstash query
input {
jdbc {
jdbc\_default\_timezone =\> "UTC"
jdbc\_driver\_class =\> "org.postgresql.Driver"
jdbc\_driver\_library =\> "/usr/share/logstash/postgresql-42.2.18.jar"
statement =\> 'SELECT
i."ItemId",
i."PharmacyId",
i."Name\_I",
i."Name\_II",
i."parentId",
i."Name\_II" as "name",
i."DoseForItem",
i."MercatoId",
id."CurrentStock",
i."ManufacturerId",
i."TouchScreenId" ,
i."ClassifyId" ,
i."DefaultUomId" ,
id."PosSelectCounter",
i."CategoryId",
i."PharmaceuticalFormId",
m."Name\_II" AS "ManufacturerName\_II",
m."Name\_I" AS "ManufacturerName\_I",
ts."Title" as "TouchScreenTitle",
ic."Name\_II" AS "CategoryName\_II",
ic."Name\_I" AS "CategoryName\_I",
ic2."Name\_II" AS "ClassifyName\_II",
ic2."Name\_I" AS "ClassifyName\_I",
pf."Name\_II" AS "PharmaceuticalFormName\_II",
pf."Name\_I" AS "PharmaceuticalFormName\_I",
(
SELECT json\_agg(distinct(it."PharmacyId"::text))::text
FROM "InvTransactions" it
WHERE it."ItemId" = i."ItemId"
) AS "activePharmacies",
(
SELECT u."UomName"::text
FROM "Uoms" u
WHERE u."UomId" = i."DefaultUomId"
) AS "uomName",
(
SELECT u2."UomId"
FROM "ItemUoms" iu
LEFT JOIN "Uoms" u2 ON iu."UomId" = u2."UomId"
WHERE iu."ItemId" = i."ItemId" limit 1
) AS "smallUomId",
(
SELECT u2."UomName"
FROM "ItemUoms" iu
LEFT JOIN "Uoms" u2 ON iu."UomId" = u2."UomId"
WHERE iu."ItemId" = i."ItemId" limit 1
) AS "smallUom",
(
SELECT json\_agg(i2."PharmacyId"::text)::text
FROM "Items" i2
WHERE i2."parentId" = i."ItemId"
) AS "ItemChilds",
(
SELECT json\_agg(b."BarcodeValue")::text
FROM "ItemBarcodes" ib
LEFT JOIN "Barcodes" b ON ib."BarcodeId" = b."BarcodeId"
WHERE ib."ItemId" = i."ItemId"
) AS "barcode",
(
SELECT json\_agg(s."Name\_II")::text
FROM "MedicineScientifics" ms
LEFT JOIN "Scientifics" s ON s."ScientificId" = ms."ScientificId"
WHERE ms."ItemId" = i."ItemId"
) AS "ScientificsName\_II",
(
SELECT json\_agg(s."Name\_I")::text
FROM "MedicineScientifics" ms
LEFT JOIN "Scientifics" s ON s."ScientificId" = ms."ScientificId"
WHERE ms."ItemId" = i."ItemId"
) AS "ScientificsName\_I",
(
SELECT json\_agg(s."ScientificId")::text
FROM "MedicineScientifics" ms
LEFT JOIN "Scientifics" s ON s."ScientificId" = ms."ScientificId"
WHERE ms."ItemId" = i."ItemId"
) AS "scientificIds",
(
SELECT json\_agg(ti."Name\_I")::text
FROM "TherapeuticIndicationItems" tii
LEFT JOIN "TherapeuticIndications" ti ON tii."TherapeuticIndicationId" = ti."TherapeuticIndicationId"
WHERE tii."ItemId" = i."ItemId"
) AS "therapeuticIndication",
(
SELECT json\_agg(ti."TherapeuticIndicationId"::text)::text
FROM "TherapeuticIndicationItems" tii
LEFT JOIN "TherapeuticIndications" ti ON tii."TherapeuticIndicationId" = ti."TherapeuticIndicationId"
WHERE tii."ItemId" = i."ItemId"
) AS "therapeuticIndicationIds"
FROM
"Items" i
LEFT JOIN
"Manufacturers" m ON m."ManufacturerId" = i."ManufacturerId"
LEFT JOIN
"TouchScreens" ts ON ts."TouchScreenId" = i."TouchScreenId"
LEFT JOIN
"ItemCategories" ic ON ic."ItemCategoryId" = i."CategoryId"
LEFT JOIN
"ItemClassifies" ic2 ON ic2."ItemClassifyId" = i."ClassifyId"
LEFT JOIN
"PharmaceuticalForms" pf ON pf."PharmaceuticalFormId" = i."PharmaceuticalFormId"
LEFT JOIN
"ItemDetails" id ON id."ItemId" = i."ItemId"'
jdbc\_paging\_enabled =\> true
jdbc\_page\_size =\> 10000
schedule =\> "\* \* \* \* \*" # Run every minute (adjust as needed)
}
}
---
### Author:  [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [June 21, 2025, 11:33am UTC](https://discuss.elastic.co/t/logstash-not-sends-all-data-to-es/379380/2 "2025-06-21T11:33:45Z")
Welcome to the community!
Add `id` for `jdbc` you check LS statistics, what is in and out from pipeline.Also add IDs to other plugins to know which consume what. Not sure are you using anything in the filter.
```auto
input {
jdbc {
jdbc_default_timezone => "UTC"
jdbc_driver_class => "org.postgresql.Driver"
jdbc_driver_library => "/usr/share/logstash/postgresql-42.2.18.jar"
statement => ...
id => "postgresql"
}
}
```
You result should be:
```auto
"events": {
->"in": 216610
->"out": 216485,
"filtered": 216485,
"duration_in_millis": 365495,
"queue_push_duration_in_millis": 342466
},
"plugins": {
"inputs": [
{
"id": "postgresql",
"flow": {
"throughput": {
"current": 603.1,
"lifetime": 590.7
}
},
"name": "jdbc",
"events": {
"out": 216485,
"queue_push_duration_in_millis": 342466
}
}
],
```
You can use this JSON statistics, just enable xpack.monitoring settings in logstash.yml or use agent/metricbeat.
Can you provide more info:
- Which LS/ES version do you use
- LS in-memory or persistent queue?
- Is your jdbc select is equal to query which count 262K records?
- Is there any other transformations in filter? IFs, drop, ...