Skip to content

Conversation

@0xgouda
Copy link
Collaborator

@0xgouda 0xgouda commented Nov 17, 2025

  • Add new admin.maintain_tables() SQL function that abstracts the maintenance logic from the Go code
    by handling the fetch of all metric tables, updating them one by one, and deleting entries for dropped tables.

- move individual metric update logic to `admin.update_listing_table()`.
- move dropped tables removal logic to `admin.remove_dropped_tables_listing()`.
- use these new functions in the maintenance routine.
@0xgouda 0xgouda requested a review from pashagolub November 17, 2025 03:50
@0xgouda 0xgouda added enhancement New feature or request refactoring Something done as it should've been done from the start sinks Where and how to store monitored data labels Nov 17, 2025
@pashagolub pashagolub force-pushed the make-maintenance-in-plpgsql branch from 5c6510c to c245c16 Compare November 17, 2025 16:39
@coveralls
Copy link

coveralls commented Nov 17, 2025

Pull Request Test Coverage Report for Build 19766198703

Details

  • 3 of 4 (75.0%) changed or added relevant lines in 1 file are covered.
  • 1 unchanged line in 1 file lost coverage.
  • Overall coverage decreased (-0.09%) to 73.556%

Changes Missing Coverage Covered Lines Changed/Added Lines %
internal/sinks/postgres.go 3 4 75.0%
Files with Coverage Reduction New Missed Lines %
internal/sinks/postgres.go 1 74.25%
Totals Coverage Status
Change from base Build 19765583712: -0.09%
Covered Lines: 3616
Relevant Lines: 4916

💛 - Coveralls

@pashagolub pashagolub self-assigned this Nov 24, 2025
@pashagolub pashagolub force-pushed the make-maintenance-in-plpgsql branch from c245c16 to ef84dfc Compare November 26, 2025 10:03
@pashagolub
Copy link
Collaborator

I want go part look like this:

func (pgw *PostgresWriter) MaintainUniqueSources() {
	sql := "SELECT admin.maintain_tables() WHERE pg_try_advisory_lock(1571543679778230000)"
	logger := log.GetLogger(pgw.ctx)
	logger.Info("Starting maintainence...")
	if _, err := pgw.sinkDb.Exec(pgw.ctx, sql); err != nil {
		logger.Error("Maintaining measurement tables failed:", err)
	}
}

- It increases the abstraction of the maintenance logic from the Go code
by handling internally the fetch of all metric tables and updating them
one by one and deleting entries for dropped tables.

- To achieve so it uses the same functions used before in the Go code
@0xgouda 0xgouda force-pushed the make-maintenance-in-plpgsql branch from ef84dfc to 1e0053a Compare November 26, 2025 22:19
@0xgouda 0xgouda requested a review from pashagolub November 26, 2025 22:21
@0xgouda
Copy link
Collaborator Author

0xgouda commented Nov 26, 2025

I want go part look like this:

done, please review.

run: |
go generate ./api/pb/
go test -failfast -v -timeout=300s -p 1 -coverprofile=profile.cov ./cmd/... ./internal/...
go test -failfast -v -timeout=600s -p 1 -coverprofile=profile.cov ./cmd/... ./internal/...
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Increased test timeouts to 10 minutes because Test_MaintainUniqueSources_DeleteOldPartitions/MaintainUniqueSources now takes more than 2 minutes alone, due to calls to PG_SLEEP(60) in MaintainUniqueSources().

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request refactoring Something done as it should've been done from the start sinks Where and how to store monitored data

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants