Forward port connection_test configuration option from v2.x

This commit is contained in:
Karl Southern 2016-08-28 21:48:06 +01:00
parent 76e0f439a0
commit 37631a62b7
4 changed files with 25 additions and 9 deletions

View File

@ -1,13 +1,23 @@
# Change Log # Change Log
All notable changes to this project will be documented in this file, from 0.2.0. All notable changes to this project will be documented in this file, from 0.2.0.
## [1.0.0-pre] - UNRELEASED ## [0.3.1] = 2016-08-28
- Test coverage extended to multiple SQL engines - Adds connection_test configuration option, to prevent the connection test from occuring, allowing the error to be suppressed.
- Change: Timestamps are sent to SQL without timezone (See https://github.com/theangryangel/logstash-output-jdbc/issues/33 for justification) Useful for cockroachdb deployments. https://github.com/theangryangel/logstash-output-jdbc/issues/53
- Change: Removes jar files from repository, in favour of vendoring using jar-dependencies
- Change: Updates to logstash-api v2.0 ## [0.3.0] - 2016-07-24
- Change: Switches from slf4j-nop to log4j for HikariCP logging - Brings tests from v5 branch, providing greater coverage
- Change: Now selectively retries only a subset of SQL Exceptions - Removes bulk update support, due to inconsistent behaviour
- Plugin now marked as threadsafe, meaning only 1 instance per-Logstash
- Raises default max_pool_size to match the default number of workers (1 connection per worker)
## [0.2.10] - 2016-07-07
- Support non-string entries in statement array
- Adds backtrace to exception logging
## [0.2.9] - 2016-06-29
- Fix NameError exception.
- Moved log_jdbc_exception calls
## [0.2.7] - 2016-05-29 ## [0.2.7] - 2016-05-29
- Backport retry exception logic from v5 branch - Backport retry exception logic from v5 branch

View File

@ -43,6 +43,7 @@ For development:
| driver_auto_commit | Boolean | If the driver does not support auto commit, you should set this to false | No | True | | driver_auto_commit | Boolean | If the driver does not support auto commit, you should set this to false | No | True |
| driver_jar_path | String | File path to jar file containing your JDBC driver. This is optional, and all JDBC jars may be placed in $LOGSTASH_HOME/vendor/jar/jdbc instead. | No | | | driver_jar_path | String | File path to jar file containing your JDBC driver. This is optional, and all JDBC jars may be placed in $LOGSTASH_HOME/vendor/jar/jdbc instead. | No | |
| connection_string | String | JDBC connection URL | Yes | | | connection_string | String | JDBC connection URL | Yes | |
| connection_test | Boolean | Run a JDBC connection test. Some drivers do not function correctly, and you may need to disable the connection test to supress an error. Cockroach with the postgres JDBC driver is such an example. | No | Yes |
| username | String | JDBC username - this is optional as it may be included in the connection string, for many drivers | No | | | username | String | JDBC username - this is optional as it may be included in the connection string, for many drivers | No | |
| password | String | JDBC password - this is optional as it may be included in the connection string, for many drivers | No | | | password | String | JDBC password - this is optional as it may be included in the connection string, for many drivers | No | |
| statement | Array | An array of strings representing the SQL statement to run. Index 0 is the SQL statement that is prepared, all other array entries are passed in as parameters (in order). A parameter may either be a property of the event (i.e. "@timestamp", or "host") or a formatted string (i.e. "%{host} - %{message}" or "%{message}"). If a key is passed then it will be automatically converted as required for insertion into SQL. If it's a formatted string then it will be passed in verbatim. | Yes | | | statement | Array | An array of strings representing the SQL statement to run. Index 0 is the SQL statement that is prepared, all other array entries are passed in as parameters (in order). A parameter may either be a property of the event (i.e. "@timestamp", or "host") or a formatted string (i.e. "%{host} - %{message}" or "%{message}"). If a key is passed then it will be automatically converted as required for insertion into SQL. If it's a formatted string then it will be passed in verbatim. | Yes | |

2
Vagrantfile vendored
View File

@ -10,7 +10,7 @@ Vagrant.configure(2) do |config|
sed -i 's/main/main contrib non-free/g' /etc/apt/sources.list sed -i 's/main/main contrib non-free/g' /etc/apt/sources.list
apt-get update apt-get update
apt-get remove openjdk-7-jre-headless -y -q apt-get remove openjdk-7-jre-headless -y -q
apt-get install git openjdk-8-jre -y -q apt-get install git openjdk-8-jre curl -y -q
gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
curl -sSL https://get.rvm.io | bash -s stable --ruby=jruby-1.7 curl -sSL https://get.rvm.io | bash -s stable --ruby=jruby-1.7
usermod -a -G rvm vagrant usermod -a -G rvm vagrant

View File

@ -83,6 +83,9 @@ class LogStash::Outputs::Jdbc < LogStash::Outputs::Base
# Suitable for configuring retryable custom JDBC SQL state codes. # Suitable for configuring retryable custom JDBC SQL state codes.
config :retry_sql_states, validate: :array, default: [] config :retry_sql_states, validate: :array, default: []
# Run a connection test on start.
config :connection_test, validate: :boolean, default: true
# Maximum number of sequential failed attempts, before we stop retrying. # Maximum number of sequential failed attempts, before we stop retrying.
# If set to < 1, then it will infinitely retry. # If set to < 1, then it will infinitely retry.
# At the default values this is a little over 10 minutes # At the default values this is a little over 10 minutes
@ -144,10 +147,12 @@ class LogStash::Outputs::Jdbc < LogStash::Outputs::Base
validate_connection_timeout = (@connection_timeout / 1000) / 2 validate_connection_timeout = (@connection_timeout / 1000) / 2
return unless @connection_test
# Test connection # Test connection
test_connection = @pool.getConnection test_connection = @pool.getConnection
unless test_connection.isValid(validate_connection_timeout) unless test_connection.isValid(validate_connection_timeout)
@logger.error('JDBC - Connection is not valid. Please check connection string or that your JDBC endpoint is available.') @logger.error('JDBC - Connection is not reporting as validate. Either connection is invalid, or driver is not getting the appropriate response.')
end end
test_connection.close test_connection.close
end end