Extend query config with query.masked config with Type.PASSWORD for secure query masking #1593
Conversation
…ecure query masking
|
🎉 All Contributor License Agreements have been signed. Ready to merge. |
| orderInGroup = defineIncrementTimestampConfigs(config, orderInGroup); | ||
| orderInGroup = defineQueryAndQuoteConfigs(config, orderInGroup); | ||
| defineTransactionAndRetryConfigs(config, orderInGroup); | ||
| } |
There was a problem hiding this comment.
Extend this Options method, because of falling to checkstyle error of exceeding 128 lines in a method.
|
|
||
| return true; | ||
| } | ||
|
|
There was a problem hiding this comment.
Added a validation in order to limit using of either of query or query.masked config.
| String msg = | ||
| "Do not specify table filtering configs with 'query' or 'query.masked'. " | ||
| + "Remove table.whitelist / table.blacklist / table.include.list / " | ||
| + "table.exclude.list."; |
There was a problem hiding this comment.
Either of table.filtering.options or query should be provided as the connector would work in table.mode or query.mode and not both.
There was a problem hiding this comment.
Few things @KGaneshDatta
- Since query was never exposed on cloud before this, we should perform some manual testing to understand/validate the behaviour. (How frequently will the query be executed, What if the query is taking too long to exceute ? do we have any observability into this ? Can multiple queries be provided (
select * from table1;select* from table2;)) - If user has specified both table include/exclude list and query, we don't know which way they wanna go. The validation exception message currently instructs them to remove table include/exclude configs. Which might be other way round also.
| private boolean hasAnyQueryConfig() { | ||
| String query = config.getString(JdbcSourceConnectorConfig.QUERY_CONFIG); | ||
| Password queryMasked = | ||
| config.getPassword(JdbcSourceConnectorConfig.QUERY_MASKED_CONFIG); |
There was a problem hiding this comment.
If you make config.getQuery return optional, then this hasAnyQueryConfig method will not be needed.
|
Problem
Type.Passwordto ensure secure logging by masking sensitive values and preventing accidental exposure.table.modeorquery.mode.Solution
Does this solution apply anywhere else?
If yes, where?
Test Strategy
Testing done:
Release Plan