AWS Glue Workflow failed. Unable to verify existence of default database.

[thomelane]

When running:

glue.wait_for_workflow_finished(config.GLUE_WORKFLOW, glue_run_id)

Get the following error:

Exception: AWS Glue Workflow failed. Check the workflow logs on the console.

After checking the AWS Glue Workflow, the error is on the Job stage.

Clicking on the Job shows the following error:

AnalysisException: 'java.lang.RuntimeException: MetaException(message:Unable to verify existence of default database: com.amazonaws.services.glue.model.AccessDeniedException: Please migrate your Catalog to enable access to this database (Service: AWSGlue; Status Code: 400; Error Code: AccessDeniedException; Request ID: b5f4516c-0451-4a55-a015-b64f7761e21f));'

[thomelane]

See Upgrading to the AWS Glue Data Catalog Step-by-Step.

"If you created databases and tables using Athena or Amazon Redshift Spectrum prior to a region's support for AWS Glue, you can upgrade Athena to use the AWS Glue Data Catalog."

Going to Amazon Athena Console, my account had the following notification:

Clicking 'here to upgrade' solved the issue, after following along with the instructions.