@ebuildy, I already used one REST API that is Postman from Google. bulk API is like that or it have more features than postman? I have to update the server everyday automatically. Is it possible in bulk API?
Actually I'm new to logstash. As you said, I'm reading for logstash with jdbc input plugin. Would you please suggest me any example with demo about the same.
The jdbc input documentation contains an example that should get you started. If it's unclear please ask a specific question, including your current configuration, what happens, and what you expected to happen.
When I start the logstash.bat getting the following error
io/console not supported; tty will not be manipulated
I'm using: elasticsearch-2.1.1 and logstash-2.1.1
My system info:
C:\Users\Croos>systeminfo
OS Name: Microsoft Windows 8.1 Pro
OS Version: 6.3.9600 N/A Build 9600
OS Manufacturer: Microsoft Corporation
OS Configuration: Standalone Workstation
my java version:
C:\Users\Croos>java -version
java version "1.8.0_66"
Java(TM) SE Runtime Environment (build 1.8.0_66-b18)
Java HotSpot(TM) 64-Bit Server VM (build 25.66-b18, mixed mode)
@magnusbaeck,
I have successfully export the MySql table to Elasticsearch. Now my problem is,
Frequently increasing the number of documents after every update by the jdbc-logstash, Even the the database have no changes. How to control it, or Over right the previous record by new, or Delete the previous document form the elasticsearch, avoid the duplication.. Give me some idea..
statement => "SELECT * from vicidial_log WHERE start_epoch > :sql_last_value"
use_column_value => true
tracking_column => start_epoch
Actually our main db is in mariaDB not MySQL. So when I run the logstash with above config, The following error occur.
io/console not supported; tty will not be manipulated
←[31mUnknown setting 'use_column_value' for jdbc {:level=>:error}←[0m
←[31mUnknown setting 'tracking_column' for jdbc {:level=>:error}←[0m
Error: Something is wrong with your configuration.
You may be interested in the '--configtest' flag which you can
use to validate logstash's configuration before you choose
to restart a running system.
If I run with only first line,
statement => "SELECT * from vicidial_log WHERE start_epoch > :sql_last_value"
I show the following exception:
C:\xampp\htdocs\ES_PACK\logstash-2.1.1\bin>logstash -f logstash.conf
io/console not supported; tty will not be manipulated
Settings: Default filter workers: 1
Logstash startup completed
←[33mException when executing JDBC query {:exception=>#<Sequel::DatabaseError: J
ava::ComMysqlJdbcExceptionsJdbc4::MySQLSyntaxErrorException: You have an error i
n your SQL syntax; check the manual that corresponds to your MariaDB server vers
ion for the right syntax to use near ':sql_last_value' at line 1>, :level=>:warn
}←[0m
←[33mException when executing JDBC query {:exception=>#<Sequel::DatabaseError: J
ava::ComMysqlJdbcExceptionsJdbc4::MySQLSyntaxErrorException: You have an error i
n your SQL syntax; check the manual that corresponds to your MariaDB server vers
ion for the right syntax to use near ':sql_last_value' at line 1>, :level=>:warn
}←[0m
Those parameters are apparently only available in recent releases of the jdbc plugin. Upgrade the plugin (preferred) or consult the documentation of the plugin you have.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.