Impossible de définir mon field comme un geo_point

[IneedHelp (Damso)]

Bonjour,
Je vais d'abord expliquer le contexte. Je développe un outil pour analyser différent type de logs et en tirer le maximum d'information. J'utilise filebeat, logstash, elactic et kibana pour la partie traitement/stockage/Interface. A ce jour mon prototype est fonctionnel sauf sur un point : il manque la carte sur laquelle afficher les coordonnées liées aux adresses IP.
J'utilise geoip pour obtenir les coordonnés gps de mes IP. Tout fonctionne très bien, j'obtiens bien toutes mes infos. Je les envois ensuite dans elastic ( là encore aucun problème ) mais quand je consulte mes données, mes coordonnés gps sont enregistrées comme des nombre et non pas des geo_point. J'ai donc les coordonnées gps pour chaque ip mais je ne peux pas les afficher sur une carte (ce n'est pas inutile mais peu parlant)
Je ne sais pas si j'ai été clair mais en résumé j'ai des coordonnés gps via geoip et je veux que logstash les traitent en tant que telles et non pas comme de simples nombres.
Du coté de mon fichier .conf ( avec les filtres grok mon fichier fait 180 lignes donc j'isole la partie qui nous intéresse)

geoip {
    source => "clientip"
}

j'ai essayé diverses choses comme :

geoip {
	source => "clientip"
}
mutate {
	add_field => {"[geoip][coordinate]" => "%{[geoip][location][lat]}"}
	add_field => {"[geoip][coordinate]" => "%{[geoip][location][lon]}"}
	convert => {
		"[geoip][coordinate]" => "float"
	}
}

geoip {
	source => "clientip"
}
mutate {
	convert => {
		"[geoip][location]" => "float"
	}
}

Bref à peu près tout ce que j'ai pu trouver comme "solution" sur les précédents topics ( Bien que la plupart renvoyaient sur la doc elastic qui ne marche pas quand je l'utilise de manière brut )
Pour ma template j'adapte en fonction de fichier confmais c'est toujours plus ou moins la même chose:

{
 "index_patterns": ["squid-*"],
 "mappings": {
	"_default_": {
		"properties": {
		  "clientip": { "type": "ip" },
		  "server_ip": { "type": "ip" },
		  "user": { "type": "keyword" },
		  "timestamp": { "type": "date" },
		  "Log_File_Format": { "type": "keyword" },
		  "Data_Sent": { "type": "number" },
		  "Data_Received": { "type": "number" },
		  "HTTP_Result": { "type": "number" },
		  "bytes": { "type": "number" },
		  "delay": { "type": "number" },
		  "facility": { "type": "number" },
		  "httpversion": { "type": "number" },
		  "pid": { "type": "number" },
		  "priority": { "type": "number" },
		  "response": { "type": "number" },
		  "windows_response": { "type": "number" },
		  "geoip": {
		    "dynamic": true,
			"properties" : {
			  "ip": { "type": "ip" },
			  "location": {"type": "geo_point"}
			}
		  }
        }
    
	}
 }
}

Désolé si je ne suis pas trop clair mais c'est la seul partie que je n'arrive pas à debuger et qui m’empêche de présenter mon prototype donc je suis un peu à bout de nerf

Merci d'avance pour votre réponse

[dadoonet (David Pilato)]

Tu peux montrer un document quand tu ne mets plus rien après

geoip {
	source => "clientip"
}

?

[IneedHelp (Damso)]

Pour un peu mieux expliquer, j'utilise filebeat pour aller récupérer les fichiers logs dans différents dossiers correspondant chacun à un format. En fonction du dossier j'applique un tags ( ex: IIS,syslog,apache, ... ), ce qui me permet d'appliquer le bon filtre grok.
J'ai mis la partie apache et nginx, les autres sont de la même structure (ex: if "syslog" in [tags] {grok{...}} )
Pour éviter toutes erreurs ave geoip, j'utilise un if qui vérifie si clientip est bien une ip ets si ce n'est pas une ip privée

input { 
	beats {
        port => "5044"
    }
}
filter {
        if "Apache" in [tags] or "Nginx" in [tags] {
	        if "Apache" in [tags] {
			mutate{
				add_field => { "Log_File_Format" => "apache" }
			}
		} else if "Nginx" in [tags]{
			mutate{
				add_field => { "Log_File_Format" => "nginx" }
			}
		}
		grok {
			match =>{ "message" => "%{COMMONAPACHELOG}%{GREEDYDATA:Message}"}
			remove_field => ["@timestamp","tags","input","log","ecs","@version","agent","host"]
		}
		if [Message] {
			grok{
				match =>{ "Message" => " \"%{GREEDYDATA:External_Link}\" \"%{GREEDYDATA:Browser_Properties}\"%{GREEDYDATA:rest}"}
				remove_field => ["Message"]
			}
		}
		mutate{
			rename => {"auth" => "user"}
			convert => {
				"bytes" => "integer"
				"response" => "integer"
				"httpversion" => "float"
			}
		}
		date{
			match => ["timestamp", "dd/MMM/yyyy:HH:mm:ss Z"]
			target => "timestamp"
		}
	}
        if ([clientip] =~ /(?:((([0-9A-Fa-f]{1,4}:){7}([0-9A-Fa-f]{1,4}|:))|(([0-9A-Fa-f]{1,4}:){6}(:[0-9A-Fa-f]{1,4}|((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){5}(((:[0-9A-Fa-f]{1,4}){1,2})|:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){4}(((:[0-9A-Fa-f]{1,4}){1,3})|((:[0-9A-Fa-f]{1,4})?:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){3}(((:[0-9A-Fa-f]{1,4}){1,4})|((:[0-9A-Fa-f]{1,4}){0,2}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){2}(((:[0-9A-Fa-f]{1,4}){1,5})|((:[0-9A-Fa-f]{1,4}){0,3}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){1}(((:[0-9A-Fa-f]{1,4}){1,6})|((:[0-9A-Fa-f]{1,4}){0,4}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(:(((:[0-9A-Fa-f]{1,4}){1,7})|((:[0-9A-Fa-f]{1,4}){0,5}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:)))(%.+)?|(?<![0-9])(?:(?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5])[.](?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5])[.](?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5])[.](?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5]))(?![0-9]))$/) {
		      if !([clientip] =~ /(?:((?<![0-9])(?:192[.]168[.](?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5])[.](?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5]))(?![0-9]))|((?<![0-9])(?:172[.](?:1[6-9]|2[0-9]|3[0-1])[.](?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5])[.](?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5]))(?![0-9]))|((?<![0-9])(?:10[.](?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5])[.](?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5])[.](?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5]))(?![0-9])))$/) {
			     geoip {
				      source => "clientip"
			     }
			     mutate {
				   add_field => {"[geoip][coordinate]" => "%{[geoip][location][lat]}"}
				   add_field => {"[geoip][coordinate]" => "%{[geoip][location][lon]}"}
				   convert => {
					    "[geoip][coordinate]" => "float"
				   }
			    }
		     }
	    }

}
output {
	if [tags] {
		stdout {}
	} else {
		elasticsearch {
			hosts => ["127.0.0.1:9200"]
			index => "squid-sadoc-%{Log_File_Format}" 
			manage_template => true
			template => "C:\logstash-7.5.0\squid_mapping.json"
		}
	}
}

Ma template:

{
 "index_patterns": ["squid-*"],
 "mappings": {
	"_default_": {
		"properties": {
		  "clientip": { "type": "ip" },
		  "server_ip": { "type": "ip" },
		  "user": { "type": "keyword" },
		  "timestamp": { "type": "date" },
		  "Log_File_Format": { "type": "keyword" },
		  "Data_Sent": { "type": "number" },
		  "Data_Received": { "type": "number" },
		  "HTTP_Result": { "type": "number" },
		  "bytes": { "type": "number" },
		  "delay": { "type": "number" },
		  "facility": { "type": "number" },
		  "httpversion": { "type": "number" },
		  "pid": { "type": "number" },
		  "priority": { "type": "number" },
		  "response": { "type": "number" },
		  "windows_response": { "type": "number" },
		  "geoip": {
		    "dynamic": true,
			"properties" : {
			  "ip": { "type": "ip" },
			  "location": {"type": "geo_point"},
			  "coordinate": {"type": "geo_point"}
			}
		  }
        }
    
	}
 }
}

Merci pour la rapidité de votre réponse.

[dadoonet (David Pilato)]

Je repose ma question.
Peux-tu supprimer cette partie et partager le document en sortie ?

		     mutate {
			   add_field => {"[geoip][coordinate]" => "%{[geoip][location][lat]}"}
			   add_field => {"[geoip][coordinate]" => "%{[geoip][location][lon]}"}
			   convert => {
				    "[geoip][coordinate]" => "float"
			   }
		    }

[IneedHelp (Damso)]

Excusez moi mais je ne comprends pas quel document vous voulez ? Je mets quelques infos en plus ci dessous au cas où mais je veux bien que vous m'expliquiez comment récupérer les infos qu'i vous faut:
Voici ce dont je parlais, elastic considère location.lat et location.lon comme des nombres mais ne veux pas considérer location comme une coordonné gps.

[dadoonet (David Pilato)]

J'ai compris ton problème.
J'essaye de t'aider à diagnostiquer ce qui ne va pas.

Pour cela, j'ai besoin d'avoir un document type une fois le filtre geoip passé.

[IneedHelp (Damso)]

Désolé de vous faire répéter mais je ne comprends pas ce qu'est un document de sortie. Je pense que c'est ce qui se rapproche le plus de ce que vous demander ( note : certaines informations sont modifiés car je ne souhaites pas les montrer (ex: adresse ip, ...) ):

{
  "_index": "squid-sadoc-ca",
  "_type": "_doc",
  "_id": "hE_T628BA7pN0dmLVI_V",
  "_version": 1,
  "_score": null,
  "_source": {
    "Port": "443",
    "timestamp": "2019-11-23T20:50:58.000Z",
    "Log_Type": "major",
    "Log_File_Format": "ca",
    "message": "major - [IPV4] - USERNAME - 2019/11/23 21:50:58 - VPNSSL02 - Requesting user to confirm access to invalid SSL site - Host: HOST, Port: 443, Request: GET URL HTTP/1.1",
    "clientip": "IPV4",
    "url": "URL ",
    "geoip": {
      "country_name": "France",
      "longitude": -0.6045,
      "latitude": 44.9504,
      "postal_code": "33290",
      "continent_code": "EU",
      "city_name": "Parempuyre",
      "country_code3": "FR",
      "location": {
        "lat": 44.9504,
        "lon": -0.6045
      },
      "ip": "IPV4",
      "region_code": "33",
      "timezone": "Europe/Paris",
      "country_code2": "FR",
      "region_name": "Gironde"
    },
    "Connexion_Type": "VPNSSL02",
    "Request_Method": "GET",
    "Host": "HOST",
    "user": "USERNAME",
    "Message": "Requesting user to confirm access to invalid SSL site"
  },
  "fields": {
    "timestamp": [
      "2019-11-23T20:50:58.000Z"
    ]
  },
  "highlight": {
    "Log_Type.keyword": [
      "@kibana-highlighted-field@major@/kibana-highlighted-field@"
    ]
  },
  "sort": [
    1574542258000
  ]
}

Si cela ne correspond pas, pouvez me préciser comment récupérer l'information que vous demander.

Merci d'avance.

[ahmed_charafouddine]

 "geoip": {
  "country_name": "France",
  "longitude": -0.6045,
  "latitude": 44.9504,
  "postal_code": "33290",
  "continent_code": "EU",
  "city_name": "Parempuyre",
  "country_code3": "FR",
  "location": {
    "lat": 44.9504,
    "lon": -0.6045
  }

Mais vous avez bien les coordonnées Geo de IP.

[ahmed_charafouddine]

@IneedHelp

quand je consulte mes données, mes coordonnés gps sont enregistrées comme des nombre et non pas des geo_point.

Geo_point n'est pas un type.

@IneedHelp

[..] je veux que logstash les traitent en tant que telles et non pas comme de simples nombres. [...]

ça veut dire quoi ?

[IneedHelp (Damso)]

Effectivement j'ai bien les coordonnés liées à l'adresse IP qui m'intéresse mais quand je souhaite les afficher sur une carte kibana me dis qu'il n'y a pas de geo_point dans mon index.

[dadoonet (David Pilato)]

Ok. Donc geoip.location a bien une structure compatible avec le geo_point datatype.

Peux-tu donc maintenant vérifier quel est le mapping de ton index.

GET squid-sadoc-ca/_mapping

[IneedHelp (Damso)]

{
  "mapping": {
    "properties": {
      "Connexion_Type": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Data_Sent": {
        "type": "long"
      },
      "Data_received": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "HTTP_Result": {
        "type": "long"
      },
      "Host": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Log_File_Format": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Log_Type": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Message": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Port": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Request": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Request_Method": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Request_Status": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Request_method": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "clientip": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "delay": {
        "type": "long"
      },
      "geoip": {
        "properties": {
          "city_name": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "continent_code": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "country_code2": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "country_code3": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "country_name": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "ip": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "latitude": {
            "type": "float"
          },
          "location": {
            "properties": {
              "lat": {
                "type": "float"
              },
              "lon": {
                "type": "float"
              }
            }
          },
          "longitude": {
            "type": "float"
          },
          "postal_code": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "region_code": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "region_name": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "timezone": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      },
      "message": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "server_ip": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "timestamp": {
        "type": "date"
      },
      "url": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "user": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      }
    }
  }
}

[dadoonet (David Pilato)]

Ton mapping est incorrect.

Tu dois donc supprimer l'index et recommencer afin de bénéficier du template que tu as créé.

Pour vérifier qu'il est bien appliqué, tu peux faire ceci. Attention, tes données existantes seront perdues.

DELETE squid-sadoc-ca
PUT squid-sadoc-ca
GET squid-sadoc-ca/_mapping

Si le mapping n'est toujours pas bon c'est que ton template ne s'applique pas correctement. Si le mapping est correct, ma théorie est que tu as fait le template après avoir inséré des premières données. Or le template n'est appliqué qu'à la création de l'index.

[IneedHelp (Damso)]

Malheureusement non à chaque modification de mon template j'ai supprimé l'index et les données je pense donc que le template s'applique mal...

[dadoonet (David Pilato)]

Dans le template, change _default_ par _doc pour voir.

[IneedHelp (Damso)]

Pas de changement

{
  "mapping": {
    "properties": {
      "Connexion_Type": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Data_Sent": {
        "type": "long"
      },
      "Data_received": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "HTTP_Result": {
        "type": "long"
      },
      "Host": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Log_File_Format": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Log_Type": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Message": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Port": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Request": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Request_Method": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Request_Status": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Request_method": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "clientip": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "delay": {
        "type": "long"
      },
      "geoip": {
        "properties": {
          "city_name": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "continent_code": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "country_code2": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "country_code3": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "country_name": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "ip": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "latitude": {
            "type": "float"
          },
          "location": {
            "properties": {
              "lat": {
                "type": "float"
              },
              "lon": {
                "type": "float"
              }
            }
          },
          "longitude": {
            "type": "float"
          },
          "postal_code": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "region_code": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "region_name": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "timezone": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      },
      "message": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "server_ip": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "timestamp": {
        "type": "date"
      },
      "url": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "user": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      }
    }
  }
}

[dadoonet (David Pilato)]

Je ne comprends pas comment ton template peut fonctionner.

Quand j'essaye ça (en 7.5.2):

PUT _template/squid
{
  "index_patterns": [
    "squid-*"
  ],
  "mappings": {
    "properties": {
      "clientip": {
        "type": "ip"
      },
      "server_ip": {
        "type": "ip"
      },
      "user": {
        "type": "keyword"
      },
      "timestamp": {
        "type": "date"
      },
      "Log_File_Format": {
        "type": "keyword"
      },
      "Data_Sent": {
        "type": "number"
      },
      "Data_Received": {
        "type": "number"
      },
      "HTTP_Result": {
        "type": "number"
      },
      "bytes": {
        "type": "number"
      },
      "delay": {
        "type": "number"
      },
      "facility": {
        "type": "number"
      },
      "httpversion": {
        "type": "number"
      },
      "pid": {
        "type": "number"
      },
      "priority": {
        "type": "number"
      },
      "response": {
        "type": "number"
      },
      "windows_response": {
        "type": "number"
      },
      "geoip": {
        "dynamic": true,
        "properties": {
          "ip": {
            "type": "ip"
          },
          "location": {
            "type": "geo_point"
          },
          "coordinate": {
            "type": "geo_point"
          }
        }
      }
    }
  }
}

J'ai:

{
  "error": {
    "root_cause": [
      {
        "type": "mapper_parsing_exception",
        "reason": "No handler for type [number] declared on field [pid]"
      }
    ],
    "type": "mapper_parsing_exception",
    "reason": "Failed to parse mapping [_doc]: No handler for type [number] declared on field [pid]",
    "caused_by": {
      "type": "mapper_parsing_exception",
      "reason": "No handler for type [number] declared on field [pid]"
    }
  },
  "status": 400
}

Après une modification, j'obtiens:

DELETE _template/squid
PUT _template/squid
{
  "index_patterns": [
    "squid-*"
  ],
  "mappings": {
    "properties": {
      "clientip": {
        "type": "ip"
      },
      "server_ip": {
        "type": "ip"
      },
      "user": {
        "type": "keyword"
      },
      "timestamp": {
        "type": "date"
      },
      "Log_File_Format": {
        "type": "keyword"
      },
      "Data_Sent": {
        "type": "long"
      },
      "Data_Received": {
        "type": "long"
      },
      "HTTP_Result": {
        "type": "long"
      },
      "bytes": {
        "type": "long"
      },
      "delay": {
        "type": "long"
      },
      "facility": {
        "type": "long"
      },
      "httpversion": {
        "type": "long"
      },
      "pid": {
        "type": "long"
      },
      "priority": {
        "type": "long"
      },
      "response": {
        "type": "long"
      },
      "windows_response": {
        "type": "long"
      },
      "geoip": {
        "dynamic": true,
        "properties": {
          "ip": {
            "type": "ip"
          },
          "location": {
            "type": "geo_point"
          },
          "coordinate": {
            "type": "geo_point"
          }
        }
      }
    }
  }
}
DELETE squid-1
PUT squid-1
GET squid-1/_mapping

Qui donne:

{
  "squid-1" : {
    "mappings" : {
      "properties" : {
        "Data_Received" : {
          "type" : "long"
        },
        "Data_Sent" : {
          "type" : "long"
        },
        "HTTP_Result" : {
          "type" : "long"
        },
        "Log_File_Format" : {
          "type" : "keyword"
        },
        "bytes" : {
          "type" : "long"
        },
        "clientip" : {
          "type" : "ip"
        },
        "delay" : {
          "type" : "long"
        },
        "facility" : {
          "type" : "long"
        },
        "geoip" : {
          "dynamic" : "true",
          "properties" : {
            "coordinate" : {
              "type" : "geo_point"
            },
            "ip" : {
              "type" : "ip"
            },
            "location" : {
              "type" : "geo_point"
            }
          }
        },
        "httpversion" : {
          "type" : "long"
        },
        "pid" : {
          "type" : "long"
        },
        "priority" : {
          "type" : "long"
        },
        "response" : {
          "type" : "long"
        },
        "server_ip" : {
          "type" : "ip"
        },
        "timestamp" : {
          "type" : "date"
        },
        "user" : {
          "type" : "keyword"
        },
        "windows_response" : {
          "type" : "long"
        }
      }
    }
  }
}

[IneedHelp (Damso)]

Merci pour votre investissement mais malheureusement du côté de "location", rien ne change

{
  "mapping": {
    "properties": {
      "Connexion_Type": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Data_Sent": {
        "type": "long"
      },
      "Data_received": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "HTTP_Result": {
        "type": "long"
      },
      "Host": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Log_File_Format": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Log_Type": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Message": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Port": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Request": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Request_Method": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Request_Status": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Request_method": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "clientip": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "delay": {
        "type": "long"
      },
      "geoip": {
        "properties": {
          "city_name": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "continent_code": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "country_code2": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "country_code3": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "country_name": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "ip": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "latitude": {
            "type": "float"
          },
          "location": {
            "properties": {
              "lat": {
                "type": "float"
              },
              "lon": {
                "type": "float"
              }
            }
          },
          "longitude": {
            "type": "float"
          },
          "postal_code": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "region_code": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "region_name": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "timezone": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      },
      "message": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "server_ip": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "timestamp": {
        "type": "date"
      },
      "url": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "user": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      }
    }
  }
}

[dadoonet (David Pilato)]

Peux-tu partager les commandes exactes que tu lances?

[IneedHelp (Damso)]

Pour kibana et elactic, j’exécute le fichier.bat correspondant. Pour Logstash, j'utilise la commande "bin\logstash -f filename.conf" et pour filebeat la commande "filebeat" depuis l'invite de commande windows.