# Filebeat not loading one index in Kibana Dataview

**URL:** <https://discuss.elastic.co/t/filebeat-not-loading-one-index-in-kibana-dataview/354238>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [February 27, 2024, 2:01pm UTC](https://discuss.elastic.co/t/filebeat-not-loading-one-index-in-kibana-dataview/354238 "2024-02-27T14:01:19Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![FileFile](https://avatars.discourse-cdn.com/v4/letter/f/8baadc/32.png) [@FileFile](https://discuss.elastic.co/u/FileFile)\
**Post date:** [February 27, 2024, 2:01pm UTC](https://discuss.elastic.co/t/filebeat-not-loading-one-index-in-kibana-dataview/354238/1 "2024-02-27T14:01:19Z")

</div>

Hello I have the following Index and pipeline created from a csv

```auto
{
  "mappings": {
    "_meta": {
      "created_by": "file-data-visualizer"
    },
    "properties": {
      "@timestamp": {
        "type": "date"
      },
      "DESCRIPCION": {
        "type": "text"
      },
      "FAMILIA": {
        "type": "keyword"
      },
      "MARCA": {
        "type": "keyword"
      },
      "REFERENCIA": {
        "type": "keyword"
      },
      "STOCK": {
        "type": "long"
      },
      "STOCK MINIMO": {
        "type": "keyword"
      }
    }
  }
}

```

```auto

[
  {
    "csv": {
      "field": "message",
      "target_fields": [
        "REFERENCIA",
        "FAMILIA",
        "DESCRIPCION",
        "MARCA",
        "STOCK",
        "STOCK MINIMO"
      ],
      "separator": ";",
      "ignore_missing": false
    }
  },
  {
    "convert": {
      "field": "STOCK",
      "type": "long",
      "ignore_missing": true
    }
  },
  {
    "remove": {
      "field": "message"
    }
  }
]

```

I have the following filebeat.yml that I execute and the rest of the indexes and pipelines work perfectly fine, but the one mentioned before does not load any data

```auto

filebeat.inputs:
- type: log
  paths:
  - C:\AA_CopiaNAS_to_Local\ELASTIC\AVD\*.csv
  exclude_lines: ['^"?CorrelationId"?,"?UserName"?,"?SessionHostName"?,"?StartTime_String"?,"?EndTime_String"?']
  tags: ["avd"]
- type: log
  paths:
  - C:\AA_CopiaNAS_to_Local\ELASTIC\VDI\*.csv
  exclude_lines: ['^"?SessionKey"?,"?Session"?,"?Machine"?,"?StartDate"?,"?EndDate"?']
  tags: ["vdi"] 
- type: log
  paths:
  - C:\AA_CopiaNAS_to_Local\ELASTIC\Aprovisionamiento\Almacen\*.csv
  exclude_lines: ['^"?REFERENCIA"?;"?FAMILIA"?;"?DESCRIPCION"?;"?MARCA"?;"?STOCK"?;"?STOCK MINIMO"?']
  tags: ["aprovalmacen"]
- type: log
  paths:
  - C:\AA_CopiaNAS_to_Local\ELASTIC\Aprovisionamiento\Almacen_ICP\*.csv
  exclude_lines: ['^"?REFERENCIA"?;"?FAMILIA"?;"?DESCRIPCION"?;"?MARCA"?;"?STOCK"?;"?STOCK MINIMO"?']
  tags: ["aprovalmacenicp"]
- type: log
  paths:
  - C:\AA_CopiaNAS_to_Local\ELASTIC\Aprovisionamiento\Envios\*.csv
  exclude_lines: ['^"?Fecha envio"?;"?INCIDENCIA/ PETICION"?;"?Stock sacado de:"?;"?SOLICITUD"?;"?Entidad"?;"?CECO"?;"?UDS"?;"?MODELO/ DESCRIPCION"?;"?FAMILIA"?;"?USUARIO FINAL"?;"?Enviar a:"?;"?PROVINCIA"?;"?LOCALIDAD"?;"?CP"?;"?DIRECCION"?;"?PERSONA DE CONTACTO"?;"?TELEFONO"?;"?Compra"?']
  tags: ["aprovenvios"]
- type: log
  paths:
  - C:\AA_CopiaNAS_to_Local\ELASTIC\Aprovisionamiento\Cestas_ARIBA\*.csv
  exclude_lines: ['^"?\[PC\] ID de pedido"?;"?\[SOCC\]Contrato \(ID de contrato\)"?;"?\[SOCC\] ID de solicitud de compra"?;"?\[SOCC\] Título de la solicitud de compra"?;"?\[PC\]Proveedor \(Proveedor común\)"?;"?\[PC\]Código de la empresa \(Código de la empresa\)"?;"?\[PC\] Número de línea del pedido de compra"?;"?\[PC\] Descripción"?;"?\[PC\]Mercancía \(ID de la mercancía\)"?;"?\[PC\]Mercancía \(Mercancía\)"?;"?\[PC\]Pieza \(Nombre de la pieza\)"?;"?\[PC\] Es un artículo de PunchOut"?;"?\[PC\]Centro de costes \(ID de centro de costes\)"?;"?\[SOCC\]Centro de costes \(Centro de costes\)"?;"?\[PC\]Unidad de medida \(Unidad de medida\)"?;"?\[PC\] Estado del pedido de compra"?;"?\[PC\]Fecha de pedido \(Fecha\)"?;"?\[PC\]Solicitante \(Usuario\)"?;"?\[SOCC\]Preparador \(Usuario\)"?;"?\[SOCC\] Entregar a"?;"?\[SOCC\]Pieza \(Número de pieza del fabricante\)"?;"?sum\(if\(Gasto de PO<=0\.0,0\.0,Cantidad de PO<=0\.0,0\.0,Gasto de PO\)\)"?;"?sum\(if\(Cantidad de PO<=0\.0,0\.0,Cantidad de PO\)\)"?.*']
  tags: ["aprovcestasariba"]
- type: log
  paths:
  - C:\AA_CopiaNAS_to_Local\ELASTIC\Aprovisionamiento\Facturacion_ARIBA\*.csv
  exclude_lines: ['^"?Doc\.compr\."?;"?Solicitante"?;"?Número de documento de documen"?;"?Soc\."?;"?Fecha doc\."?;"?Proveedor"?;"?Nombre 1"?;"? Valor neto de pedido "?;"? Importe facturado "?;"?FaF \(Indicador de Factura final\) - X"?;"?B \(Indicador de borrado en el doc\) - L \(borrado\) y S \(bloqueada posición\)"?;"?Texto breve"?;"?Código Proyecto Clarity"?;"?Código Proyecto Clarity2"?;"?Cta\.mayor"?;"?Ce\.coste"?;"?UMP"?;"?Grupo art\."?;"?II"?;"?Ctd\.pedido"?;"?Activo fijo"?;"?SNº"?;"?Fe\.entrega"?;"?Fecha Fin Contrato"?;"?Cantidad entregada"?;"?Número de posición de document"?;"?%"?;"?Código de proyecto"?;"?Nº ident\.fis\.1"?;"?Cantidad"?;"?Nombre Sociedad"?']
  tags: ["aprovfactariba"]

cloud.id: "id"
cloud.auth: "pass"

output.elasticsearch:
  indices:
  - index: "avd.prod"
    when.contains:
      tags: "avd"
  - index: "vdi.prod"
    when.contains:
      tags: "vdi"
  - index: "aprov.almacen.prod"
    when.contains:
      tags: "aprovalmacen"
  - index: "aprov.almacenicp.prod"
    when.contains:
      tags: "aprovalmacenicp"
  - index: "aprov.envios.prod"
    when.contains:
      tags: "aprovenvios"
  - index: "aprov.cestasariba.prod"
    when.contains:
      tags: "aprovcestasariba"
  - index: "aprov.factariba.prod"
    when.contains:
      tags: "aprovfactariba"

  pipelines:
  - pipeline: "avd.prod-pipeline"
    when.contains:
      tags: "avd"
  - pipeline: "vdi.prod-pipeline"
    when.contains:
      tags: "vdi"
  - pipeline: "aprov.almacen.prod-pipeline"
    when.contains:
      tags: "aprovalmacen"
  - pipeline: "aprov.almacenicp.prod-pipeline"
    when.contains:
      tags: "aprovalmacenicp"
  - pipeline: "aprov.envios.prod-pipeline"
    when.contains:
      tags: "aprovenvios"
  - pipeline: "aprov.cestasariba.prod-pipeline"
    when.contains:
      tags: "aprovcestasariba"
  - pipeline: "aprov.factariba.prod-pipeline"
    when.contains:
      tags: "aprovfactariba"

setup:
  template.enabled: false
  ilm.enabled: false

```

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [February 27, 2024, 4:24pm UTC](https://discuss.elastic.co/t/filebeat-not-loading-one-index-in-kibana-dataview/354238/2 "2024-02-27T16:24:16Z")

</div>

Hi @FileFile Welcome to the community.

> [@FileFile](#):
>
> but the one mentioned before does not load any data

Apologies but I don't know which one was mentioned so can you just be direct which index is not loading?

And whichever they're one there is is their data. Is there new data?

Did you look in the logs for any errors?

---

<div class="post-metadata">

**Author:** ![FileFile](https://avatars.discourse-cdn.com/v4/letter/f/8baadc/32.png) [@FileFile](https://discuss.elastic.co/u/FileFile)\
**Post date:** [February 28, 2024, 7:46am UTC](https://discuss.elastic.co/t/filebeat-not-loading-one-index-in-kibana-dataview/354238/3 "2024-02-28T07:46:37Z")

</div>

The one with the tag "aprovalmacenicp"  
I have created the index, pipeline and dataview with the upload a file option.  
I have tried to load new data but it does not.  
I don't see any error in the log

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [February 28, 2024, 1:22pm UTC](https://discuss.elastic.co/t/filebeat-not-loading-one-index-in-kibana-dataview/354238/4 "2024-02-28T13:22:00Z")

</div>

Have you tried debugging it by just taking out all the other ones and all the conditions and see that it actually works on its own?

That's what I would probably do

There might be a bug with the ingest pipeline or something else

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 27, 2024, 3:22pm UTC](https://discuss.elastic.co/t/filebeat-not-loading-one-index-in-kibana-dataview/354238/5 "2024-03-27T15:22:34Z")

</div>

Closed on March 27, 2024, 3:22pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.