-
Notifications
You must be signed in to change notification settings - Fork 65
[*] migrate maintenance task logic to PL/pgSQL #1025
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
[*] migrate maintenance task logic to PL/pgSQL #1025
Conversation
- move individual metric update logic to `admin.update_listing_table()`. - move dropped tables removal logic to `admin.remove_dropped_tables_listing()`. - use these new functions in the maintenance routine.
- the `MaintainUniqueSources` test calls `pgw.MaintainUniqueSources()` 3 times which sleeps a minute each time.
5c6510c to
c245c16
Compare
Pull Request Test Coverage Report for Build 19766198703Details
💛 - Coveralls |
c245c16 to
ef84dfc
Compare
|
I want go part look like this: func (pgw *PostgresWriter) MaintainUniqueSources() {
sql := "SELECT admin.maintain_tables() WHERE pg_try_advisory_lock(1571543679778230000)"
logger := log.GetLogger(pgw.ctx)
logger.Info("Starting maintainence...")
if _, err := pgw.sinkDb.Exec(pgw.ctx, sql); err != nil {
logger.Error("Maintaining measurement tables failed:", err)
}
} |
- It increases the abstraction of the maintenance logic from the Go code by handling internally the fetch of all metric tables and updating them one by one and deleting entries for dropped tables. - To achieve so it uses the same functions used before in the Go code
ef84dfc to
1e0053a
Compare
done, please review. |
| run: | | ||
| go generate ./api/pb/ | ||
| go test -failfast -v -timeout=300s -p 1 -coverprofile=profile.cov ./cmd/... ./internal/... | ||
| go test -failfast -v -timeout=600s -p 1 -coverprofile=profile.cov ./cmd/... ./internal/... |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Increased test timeouts to 10 minutes because Test_MaintainUniqueSources_DeleteOldPartitions/MaintainUniqueSources now takes more than 2 minutes alone, due to calls to PG_SLEEP(60) in MaintainUniqueSources().
admin.maintain_tables()SQL function that abstracts the maintenance logic from the Go codeby handling the fetch of all metric tables, updating them one by one, and deleting entries for dropped tables.