Provides various metrics for service's databases usage.
Is built on top of
Integrates nicely with Transferwise Entrypoints system.
Example dashboards:
Assuming, your data sources are using HikariCP, only thing you need to do, is to add a dependency.
runtimeOnly "com.transferwise.common:tw-entrypoints-starter"For best results, it is also recommended to integrate with TW Service Comms.
You can also use Database Access Statistics in test code, for example to verify that your hibernate magic code does not have any N+1 problem:
unitOfWorkManager.createEntryPoint("test", "test").toContext().execute({
mvcResult = getMockMvc()
.perform(post("/v1/batchPayouts/getBatchList")
.contentType(MediaType.APPLICATION_JSON)
.content(jsonConverter.fromObject(request)))
.andReturn()
dbStats = DatabaseAccessStatistics.get("payout")
assert dbStats.getTransactionalQueriesCount() == 1
assert dbStats.getNonTransactionalQueriesCount() == 0
})You would need to add a dependency for this as well:
testImplementation "com.transferwise.common:tw-entrypoints"We are using JSqlParser library to parse table names from queries.
The library is pretty good, but some services have few queries, it can not parse. Also, sometimes the parsing can take so long, that it will create latency spikes or cpu burns.
In those case, you can override/control the parsing via TasQueryParsingInterceptor and TasParsedQueryRegistry.
Example for TasQueryParsingInterceptor.
@Component
public class MyTasQueryParsingInterceptor extends DefaultTasQueryParsingInterceptor {
@Override
public InterceptResult intercept(String sql) {
if (StringUtils.startsWithIgnoreCase(sql, "SET fancy_variable TO")) {
return InterceptResult.doSkip();
}
else if (sql.equals(knownUnParseableSql)){
return InterceptResult.returnParsedQuery(new ParsedQuery()
.addOperation("insert", new ParsedQuery.SqlOperation()
// Main table should always be first, as we register "first-table" metrics by that.
.addTable("transfer")
.addTable("payout")));
}
return super.intercept(sql);
}
}Example for TasParsedQueryRegistry.
@Autowired
private TasParsedQueryRegistry registry;
public void registerBadSqls(){
registry.register(knownUnParseableSql,new ParsedQuery()
.addOperation("insert", new ParsedQuery.SqlOperation()
.addTable("transfer")
.addTable("payout")));
}In case where failed parsing will create too much logs noise, you have an option to override TasQueryParsingListener.
For example:
public class MyTasQueryParsingListener extends DefaultTasQueryParsingListener {
@Override
public void parsingFailed(String sql, Duration timeTaken, Throwable t) {
if (sql.equals(knownProblematicQuery)) {
// ignore
} else {
super.parsingFailed(sql, timeTaken, t);
}
}
}The library is automatically disabling query parsing during Flyway migration, but only when you use Spring Boot autoconfiguration to setup Flyway.
If you create the Flyway bean manually, then run the FluentConfiguration through flyway customizers.
For example.
@Bean
Flyway flyway(@Qualifier("dataSource") DataSource dataSource, List<FlywayConfigurationCustomizer> flywayConfigurationCustomizers) {
FluentConfiguration config = Flyway.configure()... ;
for (var flywayConfigurationCustomizer : flywayConfigurationCustomizers){
flywayConfigurationCustomizer.customize(config);
}
return new Flyway(config);
}When you would need to disable the query parsing for specific queries, or routine, surround those with TwContext and use TasUtils.
E.g.
TwContext.current().createSubContext().execute(() -> {
TasUtils.disableQueryParsing(TwContext.current());
jdbcTemplate.update("alter update create blah");
});Copyright 2021 TransferWise Ltd.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.