# Recuperer puis indexer des donnees via une api via logstash

**URL:** https://discuss.elastic.co/t/recuperer-puis-indexer-des-donnees-via-une-api-via-logstash/345527
**Category:** Discussions en français
**Created:** [October 22, 2023, 5:02pm UTC](https://discuss.elastic.co/t/recuperer-puis-indexer-des-donnees-via-une-api-via-logstash/345527 "2023-10-22T17:02:24Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![Phildefer](https://avatars.discourse-cdn.com/v4/letter/p/9fc348/32.png) [@Phildefer](https://discuss.elastic.co/u/Phildefer)
#### Post date: [October 22, 2023, 5:02pm UTC](https://discuss.elastic.co/t/recuperer-puis-indexer-des-donnees-via-une-api-via-logstash/345527/1 "2023-10-22T17:02:24Z")

</div>

Bonjour,  
(Je reposte au bon endroit, ce topic etait dans la section russe ...)

Je cherche à automatiser la récupération régulière (toutes les semaines) de données via une API.

- Les requêtes se présentent sous cette forme : V [https://api.acleddata.com/{data}/{command}.csv](https://api.acleddata.com/%7Bdata%7D/%7Bcommand%7D.csv)

Elles permettent de récupérer un fichier au format csv.

- Je souhaiterais faire en sorte que ce fichier soit un fichier tampon donc toujours avec le meme nom et qu'il soit écrasé à chaque fois par le suivant pour eviter l'accumulation
- ensuite j'aimerai indexer automatiquement (dans un index qui serait nommé acld\_date de l'import) les données du fichier ce que je fais manuellement d'habitude. Dans Kibana j'ai ainsi un pipeline comme ceci qui est généré après l'import :

```auto
[
  {
    "csv": {
      "field": "message",
      "target_fields": [
        "event_id_cnty",
        "event_date",
        "year",
        "time_precision",
        "disorder_type",
        "event_type",
        "sub_event_type",
        "actor1",
        "assoc_actor_1",
        "inter1",
        "actor2",
        "assoc_actor_2",
        "inter2",
        "interaction",
        "civilian_targeting",
        "iso",
        "region",
        "country",
        "admin1",
        "admin2",
        "admin3",
        "location",
        "latitude",
        "longitude",
        "geo_precision",
        "source",
        "source_scale",
        "notes",
        "fatalities",
        "tags",
        "timestamp"
      ],
      "ignore_missing": false
    }
  },
  {
    "date": {
      "field": "event_date",
      "timezone": "America/New_York",
      "formats": [
        "dd MMMM yyyy"
      ]
    }
  },
  {
    "convert": {
      "field": "fatalities",
      "type": "long",
      "ignore_missing": true
    }
  },
  {
    "convert": {
      "field": "geo_precision",
      "type": "long",
      "ignore_missing": true
    }
  },
  {
    "convert": {
      "field": "inter1",
      "type": "long",
      "ignore_missing": true
    }
  },
  {
    "convert": {
      "field": "inter2",
      "type": "long",
      "ignore_missing": true
    }
  },
  {
    "convert": {
      "field": "interaction",
      "type": "long",
      "ignore_missing": true
    }
  },
  {
    "convert": {
      "field": "iso",
      "type": "long",
      "ignore_missing": true
    }
  },
  {
    "convert": {
      "field": "latitude",
      "type": "double",
      "ignore_missing": true
    }
  },
  {
    "convert": {
      "field": "longitude",
      "type": "double",
      "ignore_missing": true
    }
  },
  {
    "convert": {
      "field": "time_precision",
      "type": "long",
      "ignore_missing": true
    }
  },
  {
    "convert": {
      "field": "year",
      "type": "long",
      "ignore_missing": true
    }
  },
  {
    "remove": {
      "field": "message"
    }
  },
  {
    "set": {
      "field": "point_location",
      "value": "{{latitude}},{{longitude}}"
    }
  }
]

```

Le mapping a appliquer est celui ci :

```auto
"mappings": {
    "_meta": {
      "created_by": "file-data-visualizer"
    },
    "properties": {
      "@timestamp": {
        "type": "date"
      },
      "actor1": {
        "type": "keyword"
      },
      "actor2": {
        "type": "keyword"
      },
      "admin1": {
        "type": "keyword"
      },
      "admin2": {
        "type": "keyword"
      },
      "admin3": {
        "type": "keyword"
      },
      "assoc_actor_1": {
        "type": "keyword"
      },
      "assoc_actor_2": {
        "type": "keyword"
      },
      "civilian_targeting": {
        "type": "keyword"
      },
      "country": {
        "type": "keyword"
      },
      "disorder_type": {
        "type": "keyword"
      },
      "event_date": {
        "type": "keyword"
      },
      "event_id_cnty": {
        "type": "keyword"
      },
      "event_type": {
        "type": "keyword"
      },
      "fatalities": {
        "type": "long"
      },
      "geo_precision": {
        "type": "long"
      },
      "inter1": {
        "type": "long"
      },
      "inter2": {
        "type": "long"
      },
      "interaction": {
        "type": "long"
      },
      "iso": {
        "type": "long"
      },
      "latitude": {
        "type": "double"
      },
      "location": {
        "type": "keyword"
      },
      "longitude": {
        "type": "double"
      },
      "notes": {
        "type": "text"
      },
      "point_location": {
        "type": "geo_point"
      },
      "region": {
        "type": "keyword"
      },
      "source": {
        "type": "text"
      },
      "source_scale": {
        "type": "keyword"
      },
      "sub_event_type": {
        "type": "keyword"
      },
      "tags": {
        "type": "text"
      },
      "time_precision": {
        "type": "long"
      },
      "timestamp": {
        "type": "date",
        "format": "epoch_second"
      },
      "year": {
        "type": "long"
      }
    }
  }
} 

```

Par contre je ne vois pas du tout comment cela est faisable avec logstash et si meme c'est possible.  
Si jamais vous avez des conseils ...  
Merci !

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 19, 2023, 5:03pm UTC](https://discuss.elastic.co/t/recuperer-puis-indexer-des-donnees-via-une-api-via-logstash/345527/2 "2023-11-19T17:03:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.