Question: Finding Average Extract refresh time for an Extract

[cpare]

We're successfully scheduling an extract refresh via Lambda and polling for success, but I would prefer to add a single "WAIT" step to our orchestration to get us closer to the completion time, and then poll from there - Can I use the API to poll for the last 10 refreshes of a specific extract? I don't see a method to list jobs for an extract, or jobs with a filter for an extract.

Hoping to get this back in a way I can get Job Number, Start DateTime, Finish DateTime so I can get the execution time in seconds and feed that into my WAIT state.

I am aware that this is available by querying the database, but in my case that's not available.

[jorwoods]

It is absolutely possible! You can filter the jobs for extract refreshes, matching that workbook/datasource, that have completed.

# /// script
# requires-python = ">=3.12"
# dependencies = [
#  "tableauserverclient>=0.38",
#  "python-dotenv",
# ]
# ///
from itertools import islice
import os

from dotenv import load_dotenv
import tableauserverclient as TSC

load_dotenv()

server = TSC.Server(os.environ["TABLEAU_SERVER"], use_server_version=True)
auth = TSC.PersonalAccessTokenAuth(os.environ["TOKEN_NAME"], os.environ["TOKEN_SECRET"], site_id=os.environ["TABLEAU_SITE"])
with server.auth.sign_in(auth):
    workbook_name = "..."
    most_recent = 10
    recent_refreshes: list[TSC.BackgroundJobItem] = list(islice(
        server.jobs.filter(
            job_type="refresh_extracts",
            # The notes section will contain the workbook and datasource name.
            notes__has=workbook_name,
            # Keep only jobs that are completed.
            status="Success",
            # Max possible page size is 1,000.
            page_size=min(most_recent, 1_000)
            # Sort the jobs returned by server.
            ).order_by("-started_at")
        , most_recent
        ))

    avg_seconds = sum((j.ended_at - j.started_at).total_seconds() for j in recent_refreshes)/len(recent_refreshes)

Edit: Added the order by as discussed in the following comments.

[jorwoods]

The only catch is that the signed in user would need site or server admin permissions to be able to read the jobs endpoint.

[cpare]

@jorwoods - Thanks so much for this, it's given me a huge push in the right direction - the only issue is the "most recent" just seems to be an unsorted record limit.

For now I pushed the limit to 1000, that should work until I investigate how to set the sort..

[jorwoods]

The QuerySet object also supports a method order_by. The TSC docs for reference.

So you should be able to add a .order_by("-started_at") to get only the most recent.